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Chapter 1 
Introduction 


Abstract In this chapter, I describe the call for the use of problem-centered instruc- 
tional approaches in science, technology, engineering, and mathematics (STEM) 
education. I note the rationale for this book—specifically that it allows me space 
to explain the theoretical background of scaffolding and to explore the theoretical 
implications of a meta-analysis of computer-based scaffolding in STEM education 
that I completed with colleagues. I also posit instructional scaffolding as an inter- 
vention that extends students’ capabilities as they engage with the central problem 
in problem-centered instructional approaches. I note the difference between one-to- 
one, peer, and computer-based scaffolding, and articulate that in this book I synthe- 
size research on computer-based scaffolding in STEM education. Finally, I outline 
the structure of the book. 


Keywords Computer-based scaffolding - Meta-analysis > Problem-centered 
instruction > Scaffolding - STEM education 


1.1 Why Write a Book on Computer-Based Scaffolding in 
STEM Education? 


In the most widely read and highly cited article of Educational Psychologist, 
Kirschner, Sweller, and Clark (2006) argued that problem-centered instructional 
approaches were ineffective due to their purported incorporation of minimal guid- 
ance. There is some truth in the argument of Kirschner et al. (2006), in that problem- 
centered instructional approaches that include no student guidance lead to weaker 
learning outcomes compared to direct instruction (Alfieri, Brooks, Aldrich, & Te- 
nenbaum, 2011; Hung, 2011). However, problem-centered models of instruction do 
incorporate strong support for student learning in the form of instructional scaffold- 
ing (Hmelo-Silver, Duncan, & Chinn, 2007; Schmidt, van der Molen, te Winkel, 
& Wijnen, 2009). Furthermore, asking if problem-centered instruction or lecture is 
more effective is not asking a productive question; rather, it is crucial to consider 
effectiveness using the metric of the learning goals one is trying to promote among 
students (Hmelo-Silver et al., 2007; Kuhn, 2007). Compared to that of lecture, the 
influence of problem-centered instruction paired with appropriate student support 
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on student learning is stronger in terms of the principles that connect concepts and 
application of learned content to new problems (Gijbels, Dochy, Van den Bossche, 
& Segers, 2005; Schmidt et al., 2009; Strobel & van Barneveld, 2009; Walker & 
Leary, 2009) and long-term retention of knowledge (Dochy, Segers, Van den Boss- 
che, & Gijbels, 2003; Kuhn, 2007; Strobel & van Barneveld, 2009). That problem- 
centered instruction fares well when it comes to deep content learning and prin- 
ciples and application outcomes is well-established. But the effectiveness of various 
computer-based scaffolding strategies is less well understood. That is the need that 
this book, and the underlying meta-analysis project, sought to address. 

While meta-analyses and meta-syntheses have established convincing evidence 
bases in support of the effectiveness of problem-centered instructional models, such 
syntheses of empirical research on instructional scaffolding are an emergent phe- 
nomenon (Belland, Walker, Kim, & Lefler, 2014; Belland, Walker, Olsen, & Leary, 
2015; Swanson & Deshler, 2003; Swanson & Lussier, 2001). Existing meta-analy- 
ses are either small-scale, or only focus on one subtype of computer-based scaffold- 
ing. For example, one such meta-analysis focuses on dynamic assessment (Swanson 
& Lussier, 2001). In another, included studies were referrals from a narrative review 
of studies on computer-based scaffolding (Belland, Walker, et al., 2015). 

Instructional scaffolding is an essential tool to support students during problem- 
centered instruction (Belland, Glazewski, & Richardson, 2008; Lu, Lajoie, & Wise- 
man, 2010; Reiser, 2004; Schmidt, Rotgans, & Yew, 2011). It makes sense to pursue 
synthesis of empirical research on computer-based scaffolding further so as to not 
“know less than we have proven,” which is often the risk that is run when accumu- 
lating hundreds of empirical studies on a topic (Glass, 1976, p. 8). 

The use of computer-based scaffolding paired with problem-centered instruction 
has emerged as a common and valued approach in science education (Crippen & 
Archambault, 2012; Lin et al., 2012), engineering education (Bamberger & Ca- 
hill, 2013; Gomez Puente, Eijck, & Jochems, 2013), and mathematics education 
(Aleven & Koedinger, 2002). To fully understand how to support students effec- 
tively in problem-centered instructional approaches, it is necessary to know the 
most promising strategies for instructional scaffolding (Belland et al., 2008; Lin 
et al., 2012; Quintana et al., 2004). The underlying base of empirical research on 
instructional scaffolding is undeniably large (Koedinger & Corbett, 2006; Lin et 
al., 2012), which makes it reasonable to synthesize the research using the tools of 
meta-analysis. In this way, one can determine which scaffolding characteristics and 
contexts of use have the biggest influence on learning outcomes. This book explores 
the role of instructional scaffolding in supporting students engaged in problem- 
centered instructional models in science, technology, engineering, and mathematics 
(STEM) education. It grew out of a project in which colleagues and I conducted a 
meta-analysis of research on computer-based scaffolding in STEM education. As a 
preview, computer-based scaffolding led to a statistically significant and substantial 
effect of g=0.46 on cognitive outcomes (Belland, Walker, Kim, & Lefler, In Press). 
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For many meta-analysts, reading the journal article in which my colleagues and 
I reported our meta-analysis is enough as it reports methodology, coding process, 
tests for heterogeneity, inter-rater reliability, and other important meta-analysis de- 
tails (Belland et al., In Press). However, as any researcher knows, the amount 
of theoretical background and practical details that one can fit into one journal paper 
is often woefully inadequate as there simply is not enough space. Writing a book al- 
lows one to have adequate space for important theoretical background and practical 
details. Thus, scaffolding designers and STEM education researchers and instruc- 
tors may find this book to be particularly useful as they consider how to design scaf- 
folding and the nature of coding categories used in the meta-analysis. Meta-analysts 
may also find the book to be useful as they consider how coding categories were 
defined in the underlying meta-analysis. 


1.2 What This Book Covers 


This book focuses on computer-based scaffolding in STEM education—its defini- 
tion and theoretical backing, how it has been applied in STEM education, evidence 
of its effectiveness, under what conditions computer-based scaffolding is most ef- 
fective, and which scaffolding characteristics lead to the strongest cognitive out- 
comes. The use of computer-based scaffolding paired with problem-centered in- 
struction is neither new to nor limited to STEM education (Belland, 2014; Brush & 
Saye, 2001; Hawkins & Pea, 1987; Rienties et al., 2012). Furthermore, researchers 
have found evidence of strong learning outcomes from the combination not only in 
STEM education but also in such subjects as social studies (Nussbaum, 2002; Saye 
& Brush, 2002), economics (Rienties et al., 2012), and English education (Lai & 
Calandra, 2010; Proctor, Dalton, & Grisham, 2007). 

While the underlying meta-analysis did not include studies from outside of 
STEM education, there is material in this book that is pertinent to scaffolding in 
education areas other than STEM. These include the conditions under which scaf- 
folding is used and the characteristics often present in scaffolding. However, find- 
ings about conditions under which scaffolding is most effective, student populations 
among whom scaffolding is used, and which scaffolding characteristics lead to the 
strongest impact on cognitive outcomes may not apply in non-STEM education set- 
tings. Further research is needed to ascertain this. Where the material is not directly 
applicable, it may suggest avenues for future research to better understand the role 
of computer-based scaffolding in education in the humanities and social sciences. 
Such future research is every bit as important as research on scaffolding in STEM 
education to the preparation of a well-rounded citizenry who is capable of thinking 
critically and creatively about problems (Guyotte, Sochacka, Costantino, Walther, 
& Kellam, 2014; Stearns, 1994). 
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1.3 Problem-Centered Instructional Approaches and STEM 


Problem-centered approaches have been growing in importance in STEM education 
(Abd-E]-Khalick et al., 2004; Carr, Bennett, & Strobel, 2012; Duschl, 2008; Nation- 
al Research Council, 2012). Such approaches can vary widely in terms of processes 
students and teachers follow and goals students pursue (Savery, 2006). For exam- 
ple, in terms of goals, in project-based learning and design-based learning, students 
are presented with the challenge of designing a product that addresses a problem 
(Doppelt, Mehalik, Schunn, Silk, & Krysinski, 2008; Kolodner et al., 2003; Krajcik 
et al., 1998). Design-based learning usually integrates science content with a focus 
on engineering design, and students need to follow an engineering design process 
to conceive of and build the product (Kolodner et al., 2003; Silk, Schunn, & Cary, 
2009). In project-based learning, design is not tied to a particular discipline (Barron 
et al., 1998; Krajcik, McNeill, & Reiser, 2008). In problem-based learning, students 
need to determine a conceptual solution to an ill-structured problem and defend it 
with appropriate argumentation (Barrows & Tamblyn, 1980; Belland et al., 2008; 
Hmelo-Silver, 2004). 

Processes used in problem-centered instructional approaches can range from 
studying similar cases to extract solution principles and to subsequently adapt such 
to address the present problem (case-based learning; see Kolodner, Owensby, & 
Guzdial, 2004; Srinivasan, Wilkes, Stevenson, Nguyen, & Slavin, 2007) to examin- 
ing a simulated patient, determining and addressing learning issues, and creating 
and defending a diagnosis (problem-based learning; see Barrows, 1985; Hmelo et 
al., 2001). While there are certainly variations in processes and goals of problem- 
centered approaches, a commonality is that at all of their cores are ill-structured 
problems (Jonassen, 2011; Savery, 2006). Ill-structured problems are problems for 
which there are more than one possible solution and many acceptable solution paths 
(Jonassen, 2000, 2011). They are the types of problems that professionals get paid 
to solve, and yet such problems are rarely included in K-12 curricula (Giere, 1990; 
Jonassen, 2011; Nersessian, 2008). Determining how to support students most ef- 
fectively during this important process has the potential to improve education’s 
capacity to prepare students to be successful in the twenty-first-century economy 
(Casner-Lotto & Barrington, 2006; Gu & Belland, 2015). 

As one might guess, addressing ill-structured problems is not easy. For every- 
one except perhaps the most advanced experts, addressing ill-structured problems 
requires the use of unfamiliar strategies and the learning and subsequent use of 
much content knowledge (Giere, 1990; Jonassen, 2011; Nersessian, 2008). How- 
ever, success at addressing authentic ill-structured problems in school is possible if 
students are provided appropriate instructional scaffolding to extend and enhance 
their capabilities as they engage with the target problems (Belland, 2010; Belland, 
Gu, Armbrust, & Cook, 2015; Hmelo-Silver et al., 2007). 
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1.4 Role of Scaffolding 


When considering problem-centered approaches to instruction, a central question 
has been how one can provide the support that students need to succeed in this en- 
vironment. One cannot expect to teach students all of the strategies and content that 
they need through lecture or other approaches ahead of students’ engagement with 
the central problem (Barrows & Tamblyn, 1980; Hmelo-Silver, 2004). Rather, sup- 
port provided to students engaging in problem-centered instructional approaches 
needs to incorporate scaffolding, defined as interactive support that leverages what 
students already know to help them meaningfully participate in and gain skill at 
tasks that are beyond their unassisted abilities (Belland, 2014; Hmelo-Silver et al., 
2007; Schmidt et al., 2011; van de Pol, Volman, & Beishuizen, 2010; Wood, Bruner, 
& Ross, 1976). Such support leverages what students can already do to help them 
accomplish things that they would not be able to do otherwise, such as solve the 
central problem, design an artifact to address the problem, or complete a project 
(See Fig. 1.1). Scaffolding can be provided by teachers, peers, or computer tools 
(Belland, 2014; Pifarre & Cobos, 2010; van de Pol et al., 2010), but implementing 
problem-centered instruction in K-12 settings requires the use of computer-based 
scaffolding due to the high student-to-teacher ratios in most K-12 schools (Crippen 
& Archambault, 2012; Saye & Brush, 2002). 

Instructional scaffolding differs from other instructional support strategies and tools 
in terms of what students are intended to get out of it, the timing of the support, and the 
form of the support. First, scaffolding needs to support current performance but also 
lead to the ability to perform the target skill independently in the future (Belland, 2014; 
Wood et al., 1976). Thus, a calculator does not qualify as a scaffold because while it 
supports current performance, it cannot be reasonably expected to help users calculate 
independently (i.e., without the use of a calculator) more effectively in the future. Sec- 
ond, scaffolding is used while students engage with an authentic/ill-structured problem 
(Belland, 2014; Collins, Brown, & Newman, 1989; Wood et al., 1976). Modeling a 
strategy, lecturing to students, or otherwise instructing about strategies or content before 
engagement with problems does not qualify as scaffolding. Third, scaffolding needs to 
(a) build off of what students already know and (b) be tied to ongoing assessment of 


Problem solution 


Design of artifact 


Meaningful 
participation 


lll-structured 
problem 


Completion of 
project 


Instructional 
scaffolding 


Fig. 1.1 The role of instructional scaffolding in solving ill-structured problems 
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student abilities (Graesser, Bowers, Hacker, & Person, 1997; van de Pol et al., 2010; 
Wood et al., 1976). Thus, simply telling students what to do or how to do it does not 
qualify as scaffolding, because the former approach does not elicit and build off of what 
students already know. Such an approach is not often tailored to students’ individual 
needs. Fourth, scaffolding needs to simplify some task elements but also retain and 
highlight the complexity of other task elements (Reiser, 2004; Simons & Ertmer, 2006). 
This is so as to make meaningful participation in the task possible, but also to focus 
student attention on the subsets of the problem that will lead to the desired learning and 
promote the type of productive struggle that is the highlight of effective scaffolding in- 
terventions (Belland, Glazewski, & Richardson, 2011; Reiser, 2004; Simons & Ertmer, 
2006). Without such struggle, productive learning from scaffolding cannot happen. 
Scaffolding can be provided by teachers, computers, or peers (Belland, 2014; 
Hawkins & Pea, 1987; Hogan & Pressley, 1997; Lutz, Guthrie, & Davis, 2006; Pi- 
farre & Cobos, 2010; van de Pol et al., 2010). Each of these scaffolding types form 
an important part of an overall scaffolding system (Belland, Gu, Armbrust, & Cook, 
2013; Helle, Tynjälä, & Olkinuora, 2006; Puntambekar & Kolodner, 2005; Saye & 
Brush, 2002). That is, the relative strengths and weaknesses of each can compensate 
for that of the others, forming a strong network of instructional support for students. 


1.5 Central Premises Behind This Book 


A central argument of this book is that a systematic synthesis of research on comput- 
er-based scaffolding across STEM education is warranted so as to allow researchers 
and instructors in different disciplines to benefit from research done in other fields. 
Three premises of the argument are (a) that it does not make sense to continually 
create from scratch scaffolding strategies when endeavoring to support students in 
new situations, (b) there is far too much empirical work on scaffolding in STEM 
fields to make sense of what works best in what circumstances without the use of 
meta-analysis or other comprehensive synthesis methods (e.g., meta-synthesis), and 
(c) it makes sense to synthesize research on scaffolding based in different theoreti- 
cal traditions and used in the context of diverse instructional approaches. I discuss 
and support these premises in the paragraphs that follow. 

Premise (a)—that it does not make sense to continually create from scratch scaf- 
folding strategies when endeavoring to support students in new situations—is sup- 
ported by needs for the creation of tools and strategies for supporting student learn- 
ing in a manner that builds off of prior research and development (Boote & Beile, 
2005; Edelson, 2002; Institute of Education Sciences, U.S. Department of Educa- 
tion, & National Science Foundation, 2013; Wang & Hannafin, 2005). The act of 
design, and the collection of data about how it works in authentic contexts, is cer- 
tainly an important contributor to the base of knowledge in a research area (Brown, 
1992; Edelson, 2002; Wang & Hannafin, 2005). Still, there is much published re- 
search on the effectiveness of various scaffolding strategies, and it is important that 
such research inform future development efforts. By engaging in a broad synthesis 
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of scaffolding research, one can synthesize lessons learned in diverse studies in 
order to form an understanding of what works in scaffolding (Borenstein, Hedges, 
Higgins, & Rothstein, 2009; Cooper, Hedges, & Valentine, 2009). Specifically, it 
can help one to obtain a relatively accurate estimate of the magnitude of the dif- 
ference in cognitive learning outcomes between control students and students who 
use scaffolding that (a) is designed to promote particular learning outcomes, (b) 
incorporates particular features, or (c) is used in particular contexts. This can then 
allow scaffolding designers to implement the most promising scaffolding features 
in the most promising contexts. 

For premise (b)—there is far too much empirical work on scaffolding in STEM 
fields to make sense of what works best in what circumstances without the use 
of meta-analysis or other comprehensive synthesis methods—the final traditional 
meta-analysis included 333 outcomes from 144 studies on computer-based scaf- 
folding in STEM education (Belland, Walker, Kim, & Lefler, In Press). Of 
note, multiple outcomes from the same study were maintained as separate outcomes 
when they were associated with differences in coded scaffolding or outcome char- 
acteristics. These studies are the ones that met our inclusion criteria and emerged 
from a much larger corpus of studies. Notably, included studies needed to have (a) a 
treatment and a control group, (b) an intervention that qualified as computer-based 
scaffolding, (c) sufficient information to calculate an effect size, and (d) cognitive 
learning outcomes. Synthesizing such a large number of research studies without 
the use of a systematic synthesis method would be difficult indeed. As a systematic 
synthesis method, meta-analysis can bring order to such a synthesis and lead to the 
generation of useful summary statistics. 

Our finding of 333 outcomes from 144 studies represents only some of the em- 
pirical research on computer-based scaffolding, as there is much research on com- 
puter-based scaffolding that does not include a control group or is qualitative, and 
there are many studies that do not include enough information to calculate an effect 
size. Rather than contact the authors for more information, the latter studies were 
excluded due to a decision that it was best to only use information included in re- 
search reports in our coding. Other reasons for exclusion included that two or more 
papers reported results from the same dataset. In that case, the paper with the most 
detail (e.g., dissertation) was included, while the paper with the least detail (e.g., 
conference proceeding or journal article) was excluded. In short, some excluded 
studies involved interventions that met the computer-based scaffolding definition, 
but were excluded based on failure to meet other inclusion criteria. Thus, the to- 
tal number of empirical studies on scaffolding in STEM education is considerably 
higher than the total number of studies included in the meta-analysis. 

Premise (c)—it makes sense to synthesize research on scaffolding grounded in 
different theoretical traditions and used in the context of diverse instructional ap- 
proaches—is supported by the fact that we applied a strict definition of scaffolding 
that focused on its use to extend student reasoning abilities while addressing an 
authentic, ill-structured problem. Thus, if the intervention in question did not fit 
that definition (e.g., was not used to extend student capabilities as they addressed 
authentic problems), it was excluded. This means that the scaffolding interventions 
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that were included in the meta-analysis were largely similar in terms of inherent 
goals of the intervention. Next, we employed a random effects model for analysis, 
which does not assume homogeneity of studies, and allows one to make inferences 
beyond the set of studies included in the meta-analysis (Cafri, Kromrey, & Bran- 
nick, 2010; Hedges & Vevea, 1998). Furthermore, we coded for characteristics on 
which scaffolding informed by the different theoretical traditions vary, such as in- 
tended learning outcome, scaffolding customization presence, and the basis of scaf- 
folding customization. In this way, we could test empirically if these characteristics 
influence cognitive outcomes. Next, while there is much variation in the processes 
of various problem-centered instructional approaches, to be included in this meta- 
analysis, students needed to address an authentic/ill-structured problem. Thus, if the 
central problem had one right solution, one right way to arrive at the solution, or did 
not relate to students’ lives, the article was excluded. 

In this book, I do not discuss extensively one-to-one or peer scaffolding, as that 
would be outside the scope. However, these scaffolding strategies are important 
elements of a comprehensive scaffolding strategy, as each has a different set of at- 
tributes that allow each scaffolding type to complement each other (Belland, 2014; 
Belland, Burdo, & Gu, 2015; Belland et al., 2013; Puntambekar & Kolodner, 2005; 
Puntambekar, Stylianou, & Goldstein, 2007; Saye & Brush, 2002). Readers who are 
interested in learning more about peer scaffolding are directed to Pata, Lehtinen, 
and Sarapuu (2006), Pifarre and Cobos (2010), Sabet, Tahriri, and Pasand (2013), 
and Yarrow and Topping (2001), and readers interested in learning more about one- 
to-one (teacher) scaffolding are directed to Belland, Burdo et al. (2015), Chi (1996), 
Jadallah et al. (2010), van de Pol et al. (2010), and Wood (2003). At a minimum, it 
is crucial to consider one-to-one scaffolding alongside computer-based scaffolding, 
as computer-based scaffolding by itself would be ineffective (McNeill & Krajcik, 
2009; Muukkonen, Lakkala, & Hakkarainen, 2005; Saye & Brush, 2002). This is 
in part due to a teacher’s ability to question student understanding and dynamically 
adjust support in a highly effective manner (Rasku-Puttonen, Eteläpelto, Häkkinen, 
& Arvaja, 2002; van de Pol, Volman, Oort, & Beishuizen, 2014), often in a far more 
effective manner than any computer-based tool can (Muukkonen et al., 2005; Saye 
& Brush, 2002). 


1.6 Structure of the Book 


This book was written with funding from a National Science Foundation grant 
project (award # 1251782) in which the current author and colleagues conducted a 
meta-analysis of computer-based scaffolding in STEM education. The goal in the 
project was to find out which scaffolding strategies lead to the strongest cognitive 
outcomes, and under what circumstances. The goal of this book is to communicate 
the theoretical background and findings of the project in a more descriptive fashion 
than a journal article format would allow. The intent is that readers gain an in- 
depth understanding of the historical and theoretical foundations of scaffolding and 
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problem-centered approaches to instruction, learn how scaffolding is applied and in 
what contexts, and see what scaffolding strategies have been the most effective and 
why. It is important to note that I see this book as only the start of a conversation 
on the effectiveness of scaffolding strategies in STEM education, as meta-analysis 
can include only certain quantitative studies and does not account for the many 
qualitative studies of scaffolding in STEM (Cooper et al., 2009; Sutton, 2009), in- 
cluding much of what emerges from design-based research approaches (Anderson 
& Shattuck, 2012; Brown, 1992; Wang & Hannafin, 2005). All empirical studies on 
computer-based scaffolding are important contributions to an understanding of the 
instructional approach, and so studies that were not included in the meta-analysis 
as well as new studies that emerge should be considered alongside project findings. 
Such consideration of other studies may lead to different conclusions about what 
makes scaffolding effective or not effective. Nonetheless, it is important to system- 
atically synthesize eligible quantitative research first, such that important trends 
can be identified and pursued further. Otherwise, one runs the risk of designing 
scaffolding based on an incomplete understanding of the most effective scaffolding 
strategies. 

The rest of the book proceeds as follows. In Chap. 2, I discuss the original and 
evolving definition of instructional scaffolding as well as the different theoretical 
bases that inform this evolution. Differences in the operationalization of the term 
scaffolding according to different theoretical bases are explored. This is supported 
by the idea that it is important to know how the definition of instructional scaffold- 
ing has expanded as its delivery mechanisms and the situations in which it is used 
have expanded. It is also crucial to understand what I mean when I use the term 
scaffolding, as the term means many things to many people (Palincsar, 1998; Pea, 
2004; Puntambekar & Hiibscher, 2005). 

In Chap. 3, I discuss the contexts in which computer-based scaffolding is used, 
including grade level (e.g., elementary school, graduate school), learner population 
characteristics (e.g., low-SES, traditional, under-represented), subject (e.g., science, 
technology), and problem-centered model with which scaffolding is used (e.g., 
problem-based learning, case-based learning). The wide range of contexts of use of 
scaffolding is important to consider as one thinks about how to apply the scaffold- 
ing metaphor in education and how scaffolding’s effectiveness varies according to 
the context in which it is used (Stone, 1998). Such wide variation in contexts of use 
can be seen to correspond with wide variations in scaffolding strategies. 

In Chap. 4, I discuss the intended learning outcomes of scaffolding as well as 
assessment strategies used to measure student learning from scaffolding. I also note 
alignment of the intended learning outcomes and assessment approaches with goals 
of STEM education as outlined in the Next Generation Science Standards. This is 
important, as instructional scaffolding has evolved to support students’ performance 
and learning of diverse skills (Puntambekar & Hiibscher, 2005). Given such an ex- 
pansion, it is important to see if scaffolding leads to different impacts according to 
the varied intended learning outcomes. 

In Chap. 5, I describe variations in scaffolding strategy, including scaffolding function 
(e.g., conceptual, metacognitive), context-specificity (i.e., context-specific or generic), 
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customization (e.g., fading, adding), and customization schedule (e.g., performance- 
based, fixed). These variations relate to some of the persistent debates in the scaffolding 
literature (Belland, 2011; Hannafin, Land, & Oliver, 1999; McNeill & Krajcik, 2009; 
McNeill, Lizotte, Krajcik, & Marx, 2006; Pea, 2004; Puntambekar & Htibscher, 2005). 
It is important to see if such variations in scaffolding strategy lead to differences in 
cognitive outcomes. 

I also note variations in effect size estimates according to the characteristics cov- 
ered in Chaps. 3—5. Notably, many of the details related to the methodology used 
in the underlying meta-analysis are not presented in this book. Interested readers 
should refer to Belland et al. (In Press). 

Finally, in Chap. 6, I conclude the book, noting lessons learned about scaffolding 
in STEM education and proposing directions for future research. 
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Chapter 2 
Instructional Scaffolding: Foundations 
and Evolving Definition 


Abstract This chapter covers the definition of instructional scaffolding, as well as 
its theoretical bases, and how those bases are reflected in computer-based scaffold- 
ing. Computer-based scaffolding is defined as a computer-based tool that extends 
and enhances student capabilities as students engage with authentic and ill-struc- 
tured tasks. Despite its original atheoretical nature, scaffolding was linked to many 
theoretical frameworks, including activity theory, Adaptive Character of Thought- 
Rational (ACT-R), and knowledge integration. This variation in theoretical frame- 
works has led to differing scaffolding strategies (e.g., fading, adding, and fading/ 
adding strategies) and overall scaffolding approaches. These are described in depth 
in this chapter. 


Keywords Activity theory - ACT-R - Adding - Computer-based scaffolding - 
Contingency - Design of scaffolding - Dynamic assessment > Fading - Fading/ 
adding - Intelligent tutoring systems > Intersubjectivity - Knowledge integration - 
One-to-one scaffolding - Peer scaffolding 


2.1 Historical Definition 


The metaphor of instructional scaffolding was originally proposed to describe how 
parents and teachers provided dynamic support to toddlers as they learned to con- 
struct pyramids with wooden blocks (Wood, Bruner, & Ross, 1976). This support 
was meant to extend students’ current abilities, meaning that even while supported, 
toddlers did the bulk of the work required to solve the problem. Scaffolding thus 
helped fill in key gaps in students’ abilities and knowledge such that they could then 
complete the task. In so doing, it simplified some task elements that were not central 
to learning to perform the skill independently, but also helped draw students’ atten- 
tion to particularly important task elements, ensuring that these elements were not 
simplified (Reiser, 2004). It also helped to enlist students’ interest in the learning 
task and sustain their engagement (Belland, Kim, & Hannafin, 2013). Scaffolding 
was meant to support toddlers temporarily as they engaged with problems, but also 
to lead to skill gain to enable independent problem-solving in the future (Collins, 
Brown, & Newman, 1989; Wood et al., 1976). 
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Scaffolding was contingent, meaning that scaffolding encompassed two key 
events that were at once iterative and interconnected—dynamic assessment of the 
child’s current performance characteristics and provision of just the right support 
(Collins et al., 1989; Tzuriel, 2000; van de Pol, Volman, & Beishuizen, 2011; Wood, 
2003). That is, determination of just the right support to be provided to students 
was always based on dynamic assessment. As dynamic assessment indicated that 
students were gaining skill and were on the path to being able to perform the task 
independently, support could be reduced (faded; Collins et al., 1989; Pea, 2004; 
Wood et al., 1976). If dynamic assessment indicated that students were struggling 
to participate meaningfully, support could be increased (added; Anderson, Matessa, 
& Lebiere, 1997; Koedinger & Aleven, 2007). 

Scaffolding also required intersubjectivity—an understanding of what success- 
ful performance of the target task would look like that was shared between the 
scaffolder and the scaffoldee (Wertsch & Kazak, 2005; Wood et al., 1976). This was 
considered necessary so that the students would themselves know when the task had 
been accomplished successfully, which is crucial to independent performance in the 
future (Mortimer & Wertsch, 2003; Wertsch & Kazak, 2005; Wood et al., 1976). In 
short, scaffolded performance leads to skill gain that can only lead to independent 
performance when a student also exhibits interdependence. 

Before proceeding further, it is important to acknowledge the lack of precision 
that has emerged in the term scaffolding as researchers used the term to describe a 
wide swath of instructional methods. This has been an often-lamented phenomenon 
(Pea, 2004; Puntambekar & Hiibscher, 2005; Stone, 1998). I did not set out to re- 
solve this debate, as that is beyond the scope of this book. Still, it is important to 
outline what the term scaffolding means for the purposes of this book. The first key 
feature that distinguishes scaffolding from other forms of instructional support is 
that it is temporary support that is provided as students are engaging with problems 
(Belland, 2014; Collins et al., 1989; Wood et al., 1976). As a corollary, support 
that is not provided as students engage with problems (e.g., it is provided before 
students engage with problems or it is provided as students listen to a lecture) is not 
scaffolding. According to this definition, one cannot give instruction to students, 
then have them engage in practice problems, and call the instructional interven- 
tion scaffolding. Support that continues indefinitely does not meet the scaffolding 
definition either, as this would not require that students gain skill so as to be able 
to perform the target task independently in the future (Collins et al., 1989; Wood et 
al., 1976). 

Next, scaffolding needs to lead to skill gain such that students can function in- 
dependently in the future (Belland, 2014; Pea, 2004; Wood et al., 1976). Hence, 
tools such as a calculator cannot be considered scaffolds because they are not meant 
to lead to learning. Rather, such tools are meant to continue to be used whenever 
users encounter a situation in which the tools are of use (e.g., finding square roots, 
dividing large numbers). To the contrary, scaffolding needs to simultaneously help 
students enhance skills and participate meaningfully in the performance of the tar- 
get skill (Belland, 2014; Wood et al., 1976). 
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Third, scaffolding not only simplifies tasks, but also highlights complexity there- 
in (Reiser, 2004; Wood et al., 1976). This is because struggling while attending to 
certain complexities inherent in a particular task can lead to robust learning (Reiser, 
2004; Simons & Ertmer, 2006). A job aid does not meet the definition of scaffolding 
already because it is not meant to lead to learning, but it also is disqualified because 
it only simplifies tasks and does not highlight complexity therein (Belland, 2014). 

Fourth, to qualify as scaffolding, students need to meaningfully participate in the 
target task and have an understanding of what success at the task means (Mahardale 
& Lee, 2013; Wood et al., 1976). If the tool does all or most of the work or if stu- 
dents do not know how to recognize successful performance of the target skill, then 
the possibility of skill gain is compromised (Chi, 1996; Pea, 2004). 


2.2 Scaffolding Elements 


Next, it is important to describe in detail the elements that contingency of scaffold- 
ing encompasses—dynamic assessment, providing just the right amount of support, 
and intersubjectivity. 


2.2.1 Dynamic Assessment 


Dynamic assessment and scaffolding customization were inextricably tied (See 
Fig. 2.1) in the original scaffolding definition (Wood et al., 1976). Dynamic assess- 
ment differs in goals and methods from traditional assessment in that it (a) aims 
at not only ascertaining the current level of performance, but also improving it, 
(b) aims at informing appropriate instructional practices, rather than simply clas- 
sification, and (c) focuses on students’ current and potential levels of performance 


Fig. 2.1 The role of dynamic 
assessment in the customiza- 
tion of scaffolding 
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(Lidz, 1995; Seethaler, Fuchs, Fuchs, & Compton, 2012; Tzuriel, 2000). For ex- 
ample, dynamic assessment can involve providing a series of prompts that each 
provide differing levels of support; the teacher can then determine the student’s cur- 
rent ability level based on what level of support was needed to enable adequate per- 
formance (Lidz, 1995; Seethaler et al., 2012). Dynamic assessment can also involve 
having students perform a task in the genre of the target task, noting their difficul- 
ties, designing tailored assistance, providing that, and assessing the student’s ability 
(Tzuriel, 2000). Dynamic assessment can also focus on eliciting the metacognitive 
processes in which students engage and comparing those to the type of metacogni- 
tion that is desired (Lidz, 1995). Within dynamic assessment, there is often also a 
focus on seeing what students can do in collaboration with others, which harkens 
back to the original definition of the zone of proximal development (Kozulin & 
Garb, 2002; Vygotsky, 1978). For example, teachers may draw student attention to 
particular concepts in questions or instructions in tests, thereby assessing students’ 
abilities to conduct the tasks embedded in the test, rather than their ability to inter- 
pret instructions (Kozulin & Garb, 2002). 

Dynamic assessment can be both a stand-alone intervention—and a highly effec- 
tive one at that (Seethaler et al., 2012; Swanson & Lussier, 2001; Tzuriel, 2000); for 
more information, see Swanson and Lussier (2001)—and the basis for adjustment 
of scaffolding (Poehner & Lantolf, 2005; van de Pol, Volman, & Beishuizen, 2010; 
Wood et al., 1976). When used as the basis for the provision of teacher scaffold- 
ing, teachers ask questions and observe student performance to determine the level 
of support that is needed and then provide support accordingly (van de Pol et al., 
2010). 

Dynamic assessment can also be used for adjustment of scaffolding that is al- 
ready being provided. In this case, teachers can determine the extent to which stu- 
dent skill is improving so as to lead to success without scaffolding, or with less 
scaffolding, and such adjustments can be made in real time. When used as the basis 
for the introduction, removal, or adjustment of computer-based scaffolding, stu- 
dents often need to respond to multiple choice questions (Koedinger & Aleven, 
2007; VanLehn, 2011). The veracity of the responses or lack thereof is then fed 
into model tracing in the intelligent tutoring system, and the level of support is 
thereby increased or reduced (Baker, Corbett, & Koedinger, 2007; Koedinger & 
Corbett, 2006; Murray, 1999). However, adjustment of computer-based scaffolding 
is often not performed on the basis of dynamic assessment, but rather on the basis 
of self-selection or a fixed schedule, especially in the case of scaffolding to support 
ill-structured problem-solving (Belland, 2011; McNeill, Lizotte, Krajcik, & Marx, 
2006; Metcalf, 1999). This results from difficulties in programming computer tools 
to dynamically assess how well students are performing in ill-structured problem- 
solving, when there are countless paths that can be taken that are equally correct. 
Self-selected or fixed customization may not fit the original definition of scaffold- 
ing customization (Belland, 2011; Wood et al., 1976). 
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2.2.2 Providing Just the Right Amount of Support 


First, providing just the right support refers to providing scaffolding support accord- 
ing to what dynamic assessment indicated was required (Wood et al., 1976). This 
can be either providing customized support generated in real time, as in one-to-one 
scaffolding (Jadallah et al., 2010; van de Pol, Volman, & Beishuizen, 2012), or pro- 
viding just the right combination of preformed scaffolding elements, as can occur 
with computer-based scaffolding (Koedinger & Corbett, 2006). 

Next, providing just the right amount of support depends upon adjustment in 
one or more of the following ways—adjustment to (a) the support strategies being 
used, (b) the subskill on which to focus next, and (c) the timing by which support 
is offered (Wood, 2003). One form of such adjustment—removing support—was 
later termed “fading” by Collins et al. (1989). In fading, the scaffolding provider 
removes or lessens the intensity of scaffolding based on dynamic assessment that 
indicates improved performance and the potential to perform well independently. 
Fading is designed to gradually transfer the responsibility for the performance of 
the target skill from the scaffold provider to the scaffold receiver (Collins et al., 
1989; van de Pol et al., 2010). For example, fading may first lead to a shift to scaf- 
folding strategies that are less supportive or directive and eventually to an absence 
of all scaffolding strategies altogether. As another example, the initial scaffolding 
strategy may help students overcome three major challenges in the target task, but 
after fading, the scaffolding strategy may only support learners in overcoming one 
or two of the challenges. Fading can also refer to a decrease in the frequency of scaf- 
folding messages. It has been proposed that fading may not be a necessary prerequi- 
site of transfer of responsibility in all cases; rather, ensuring that students maintain 
executive control of the underlying activity can lead to the transfer of responsibility 
from the scaffold to students (Belland, 2011). 

Scaffolding adjustment can also take the form of adding different types of sup- 
port or enhancing the support that was already present, this based on dynamic as- 
sessment that indicates that students are not making the necessary progress quickly 
enough to lead to independent problem-solving, or self-selection (Koedinger & 
Aleven, 2007; Koedinger & Corbett, 2006). As with fading, the exact nature of 
adding support can vary. It can manifest itself in (a) providing more scaffolding 
strategies, or more supportive ones, (b) scaffolding targeting more challenges, and/ 
or (c) exposing students to scaffolding messages more frequently (Baker et al., 
2007; Koedinger & Aleven, 2007; Murray, 1999). Adding scaffolding often hap- 
pens when students click a button indicating that they want more help (hints), as is 
the case with intelligent tutoring systems (Koedinger & Aleven, 2007; Koedinger 
& Corbett, 2006). In this case, the first time the hint button is pressed, a minimally 
supportive hint is given. The next times, successively more supportive hints are 
given each time, until a bottom-out hint is given that contains the solution (Koed- 
inger & Aleven, 2007). Such self-selection of hints can be tied to the position of the 
theoretical basis of intelligent tutoring systems—Adaptive Character of Thought 
(ACT-R)—that struggle is unproductive in learning (Anderson, 1983). In intelligent 
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tutoring systems, hints can also be provided based on performance, but this is less 
common (Koedinger & Aleven, 2007). 

Scaffolding interventions can also employ both strategies—adding and fading— 
depending on what the performance characteristics of the learner justifies (Koed- 
inger & Corbett, 2006). That is, if performance characteristics indicate that the stu- 
dent is not making sufficient progress, scaffolding can be added. If performance 
indicators indicate that the student is on the path to being able to perform the target 
skill independently, then scaffolding can be faded. This is employed by providers of 
one-to-one scaffolding (Chi, 1996; van de Pol et al., 2010), but also often by intel- 
ligent tutoring systems (Koedinger & Corbett, 2006). In the latter case, this often 
involves feedback that varies depending on the quality of students’ performance 
(adding/fading) as well as hints that are available on demand (adding; Koedinger 
& Corbett, 2006). 

Ultimately, the goal of scaffolding is that the learner not only gains the skills 
required to perform the target task independently, but also assumes responsibility 
for the task (Belland, 2014; Wood et al., 1976). In other words, scaffolding aims at 
promoting not only the capacity but also the willingness to perform complex tasks 
independently (Belland, Kim, et al., 2013). Lying beneath the surface of this aim 
are cognitive and motivational aims, neither of which, if satisfied, would be enough 
by itself to ensure success (Belland, Kim, et al., 2013; Wood et al., 1976). Perhaps 
accordingly, in its initial conceptualization, scaffolding included equal parts sup- 
port for motivation (recruitment, frustration control, and direction maintenance), 
and cognition (marking critical features, demonstration, and reduction in degrees 
of freedom; Belland, Kim, et al., 2013; Wood et al., 1976). Such support built off 
of toddlers’ existing skills and knowledge and was delivered as the toddler engaged 
with the problem. Within the example from Wood et al. (1976) in which adults 
helped infants learn to build pyramids, recruitment built off of the interest toddlers 
developed during free play with the wooden blocks prior to the application of the 
scaffolding approach. Central to the development of interest is establishing the im- 
portance of the learning activity to learning to perform the target skill (Gu, Belland, 
Weiss, Kim, & Piland, 2015). Frustration control helped keep learners invested in 
the task at hand even when they ran into the inevitable struggles that characterize 
authentic problem-solving. Direction maintenance aimed at keeping students on the 
path that would lead to solving the problem. Within marking critical features, tutors 
could point out the most critical factors to which students should attend. Demonstra- 
tion relied on students’ existing knowledge of how to put blocks together, extending 
such knowledge by showing students how to combine moves that they had already 
performed in new ways. When reducing the degrees of freedom, tutors would sim- 
plify the process such that students only need pay attention to the segment of the 
task that will lead to learning gains. Notably, all such scaffolding strategies built off 
of what students could already do, and extended such capabilities so as to enable 
more complex activity (Wood et al., 1976; Wood & Wood, 1996). 


2.2 Scaffolding Elements 23 


2.2.3 Intersubjectivity 


Also crucial to the definition of scaffolding and to the idea of transfer of respon- 
sibility was intersubjectivity, according to which students needed to recognize an 
appropriate solution to problems similar to the one being addressed before they 
would be able to perform the supported task independently (Mahardale & Lee, 
2013; Mortimer & Wertsch, 2003; Wood et al., 1976). Without intersubjectivity, 
students are said to be unable to engage in independent performance of the target 
skill (See Fig. 2.2). 

Intersubjectivity can be achieved without knowledge of how to perform the skill 
that scaffolding is intended to develop (Wertsch & Kazak, 2005). It is important to 
note that it is not required that the understanding be exactly the same, as partners in 
an activity likely hold differing perspectives, which can shape an understanding of 
a task (Rogoff & Toma, 1997). Furthermore, if the child and adult had an entirely 
identical understanding of what an appropriate solution would be to a problem simi- 
lar to that being addressed, then the child may not need scaffolding (Wertsch, 1984). 
Rather, the understanding of the task should be substantially similar between the 
scaffolding provider and the student. This was said to be crucial because students 
needed to be able to recognize when what they were doing was successful when 
they attempted the target tasks independently in the future (Mortimer & Wertsch, 
2003; Wood et al., 1976). In short, scaffolding could help students with how to ac- 
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Fig. 2.2 Exhibiting intersubjectivity and engaging in scaffolded performance as predictors of the 
ability to engage in independent performance 
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complish a given task, but was not suited to also establish the evidence that would 
indicate that an appropriate solution had been found to problems of similar types. 

Scaffolding can be provided by teachers (one-to-one scaffolding), peers (peer 
scaffolding), and computers (computer-based scaffolding) (Belland, 2014). In the 
next section, the scaffolding forms are defined and changes in the scaffolding defi- 
nition to encompass computer-based scaffolding are discussed. 


2.3 Scaffolding Forms 


Scaffolding forms include one-to-one, peer, and computer-based scaffolding (See 
Table 2.1). These are explained in depth in the subsections that follow. 


2.3.1 One-to-One Scaffolding 


One-to-one scaffolding is defined as one teacher working one-on-one with one stu- 
dent to dynamically assess the student’s current level, provide just the right amount 
of support for the student to perform and gain skill at the target task, and customize 
the support as needed until the scaffolding can be entirely removed and the student 
can take ownership (Belland, 2014; Chi, 1996; Graesser, Bowers, Hacker, & Person, 
1997; Lepper, Drake, & O’Donnell-Johnson, 1997; van de Pol et al., 2010). Within 
one-to-one scaffolding, it is helpful to think of scaffolding intentions—what the 
teacher seeks to accomplish by scaffolding—and scaffolding means—the specific 


Table 2.1 Overview of one-to-one, computer-based, and peer scaffolding 


One-to-one Computer-based Peer scaffolding 
scaffolding | scaffolding 
What is it? | One teacher working | Scaffolding function | Scaffolding support 
one-to-one with one | fulfilled by a computer | provided by peers 
student tool that can be embed- | of similar or greater 
ded into a curriculum or | ability 
a tool that students use 
when engaging with a 
problem outside of the 
system 
Among scaffolding Leads to the strongest | Is the most scalable Is the most scalable 


forms, what are its 
relative advantages? 


Among scaffolding 


forms, what are its rel- 
ative disadvantages? 


influence on learning 
outcomes 


Is the best at dynamic 
| customization 


| Least scalable 


| Least dynamic 


scaffolding form that 
still involves one-to- 


Has infinite patience one interaction 


| Scaffolding provider 
is not necessarily 
| more able 
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strategies used (Belland, 2012; van de Pol et al., 2010). One-to-one scaffolding 
intentions include recruiting, structuring tasks, direction maintenance, reducing the 
degrees of freedom, and frustration control (van de Pol et al., 2010). One-to-one 
scaffolding means include modeling, questioning, explaining, giving hints, and pro- 
viding feedback (van de Pol et al., 2010). Some of these same techniques are used 
in the context of other instructional approaches, so it is important to consider both 
intentions and means when considering one-to-one scaffolding (Belland, 2012). For 
example, to promote increased use of evidence in arguments, fourth grade teachers 
used such scaffolding means as praise and prompting for evidence, which led to en- 
hanced use of evidence by the students (Jadallah et al., 2010). To promote the con- 
sideration of the relations between different entities involved in a problem, teachers 
can prompt students to consider such relations and illustrate how to do so; this led 
elementary students to successfully consider such relations (Lin et al., 2014). In an- 
other example, teachers can use questioning and other strategies to help struggling 
first grade students learn to read; this helped such students rapidly reach grade-level 
reading proficiency (Rodgers, 2004). Praise and prompting for evidence can very 
well be used as part of another instructional approach. What makes the strategies 
examples of scaffolding has to do with the intended function of the strategy and the 
context in which it was used (e.g., to help students engaged in authentic problem- 
solving (Belland, 2012)). 

Due to its highly contingent nature, one-to-one scaffolding is generally consid- 
ered to be the ideal form of scaffolding (Belland, Burdo, & Gu, 2015; Chi, 1996; 
Graesser et al., 1997). Among scaffolding forms, it tends to lead to the highest 
effect sizes as indicated by a recent meta-analysis, which found that one-to-one 
scaffolding leads to an average effect size of 0.79, while step-based intelligent tu- 
toring systems led to an average effect size of 0.76 (VanLehn, 2011). Still, in most 
educational environments, one cannot expect all needed support to come from one- 
to-one scaffolding (Belland, Gu, Armbrust, & Cook, 2013; Muukkonen, Lakkala, & 
Hakkarainen, 2005; Puntambekar & Kolodner, 2005). Thus, it is important to focus 
one-to-one scaffolding to those areas where it is most effective and allow computer- 
based scaffolding to shoulder the lion’s share of responsibility for supporting stu- 
dents in the remainder of the areas in which students need support (Belland, Gu, et 
al., 2013; Muukkonen et al., 2005; Saye & Brush, 2002). 


2.3.2 Peer Scaffolding 


Peer scaffolding refers to the provision of scaffolding support by peers, and it lever- 
ages the strength in numbers of peers in classrooms (Davin & Donato, 2013; Pata, 
Lehtinen, & Sarapuu, 2006; Sabet, Tahriri, & Pasand, 2013). But it can also involve 
older children providing scaffolding support to younger students. For example, stu- 
dents with strong English-speaking abilities can use questioning and prompting to 
help English as a New Language students improve their English-speaking abilities 
(Angelova, Gunawardena, & Volk, 2006). In another example, third grade students 
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provided scaffolding support to help preschool students create crafts projects (Fair, 
Vandermaas-Peeler, Beaudry, & Dew, 2005). 

Peer scaffolding requires that a framework be provided that guides scaffolding 
(Belland, 2014). Such a framework can guide scaffolding providers with strategies 
to use and when to use them (Belland, 2014). The framework can be embedded 
in computer-based scaffolds. For example, students can be encouraged to provide 
feedback through the embedding of a peer feedback mechanism in computer-based 
scaffolds, as well as guidance on how to provide peer scaffolding in this way (Pi- 
farre & Cobos, 2010). Doing so can help college students regulate each other’s 
learning behavior (Pifarre & Cobos, 2010). 

Individual empirical studies indicate that peer scaffolding positively influences 
cognitive outcomes (Fair et al., 2005; Hakkarainen, 2004; Oh & Jonassen, 2007; 
Palincsar & Brown, 1984; Pifarre & Cobos, 2010) and helps students who are low in 
self-regulation successfully address the central problem (Helle, Tynjälä, Olkinuora, & 
Lonka, 2007), but to my knowledge no comprehensive meta-analysis addresses this 
form of scaffolding. One meta-analysis covers the influence of peer tutoring, finding 
that it leads to an average effect size of 0.4 (P. A. Cohen, Kulik, & Kulik, 1982). 

It is unlikely that peer scaffolding would be sufficient as a sole source of scaf- 
folding support, as similarly abled peers do not have the content or pedagogical 
expertise to be able to engage in the dynamic assessment and customization that 
is characteristic of one-to-one scaffolding (Belland, 2014). Peers also often do not 
have the patience and persistence of a computer program. Furthermore, when peer 
scaffolding providers are at the same grade and ability level as the peer scaffolding 
receivers, one may question the capacity for strong scaffolding interactions. How- 
ever, research on the influence of content expertise of tutors on learning outcomes 
in problem-based learning is often contradictory (Albanese, 2004; Dolmans et al., 
2002). A recent meta-analysis indicated that student learning decreases as tutor ex- 
pertise increases (Leary, Walker, Shelton, & Fitt, 2013). 


2.3.3 Computer-Based Scaffolding 


One-to-one scaffolding is a very effective method. A recent meta-analysis found 
that it leads to an average effect size of 0.79 on cognitive learning outcomes (Van- 
Lehn, 2011), which is classified as a large effect size according to J. Cohen’s (1969) 
guidelines. But it was clear that one teacher in a classroom of 30 students would not 
likely be able to provide all of the scaffolding support that her students would need 
(Saye & Brush, 2002; Tabak, 2004). Thus, computer-based scaffolding emerged as 
a tool to help share in the burden of scaffolding (Hawkins & Pea, 1987). 
Computer-based scaffolding can be defined as computer-based support that helps 
students engage in and gain skill at tasks that are beyond their unassisted abilities 
(Belland, 2014; Hannafin, Land, & Oliver, 1999; Quintana et al., 2004). Specifical- 
ly, it assists students as they generate solutions to complex, ill-structured problems 
and is provided entirely by a computer-based tool. This means that the tool helps 
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extend student capabilities such that they are able to perform at a higher level than 
they would have otherwise. For example, Belvedere invites students to articulate 
important concepts that interrelate in the problem and diagram and characterize 
links among these concepts through concept mapping (Cavalli-Sforza, Weiner, & 
Lesgold, 1994; Cho & Jonassen, 2002). 

The exact nature of support in computer-based scaffolding varies according 
to the theoretical framework—e.g., cultural historical activity theory, ACT-R, or 
knowledge integration—on which the scaffolding is based. Support created accord- 
ing to the activity theory framework is designed to stretch student abilities and 
foster the kind of struggle that the framework holds leads to learning (Akhras & 
Self, 2002; Belland & Drake, 2013; Jonassen & Rohrer-Murphy, 1999; Reiser, 
2004). Computer-based scaffolding created according to the ACT-R framework is 
designed to help students apply declarative knowledge in the context of problems 
such that they can develop production rules with which to use the target knowledge 
in the context of solving new problems (Koedinger & Corbett, 2006; VanLehn, 
2011). Such scaffolding is designed so as to help students avoid struggle, which 
ACT-R posits as inconducive to learning (Anderson, 1996). Computer-based scaf- 
folding designed according to the knowledge integration framework aims to help 
students build integrated mental models while they engage with problems (Clark 
& Linn, 2013; Linn, Clark, & Slotta, 2003). Computer-based scaffolding is largely 
less contingent than one-to-one scaffolding, although, in general, scaffolding em- 
bedded in intelligent tutoring systems is more contingent than other computer-based 
scaffolding. 

Recent smaller-scale meta-analyses showed that computer-based scaffolding led 
to average effect sizes of 0.53 (Belland, Walker, Olsen, & Leary, 2015) and 0.44 
(Belland, Walker, Kim, & Lefler, 2014). In the meta-analysis from which this book 
grew, the average effect size of computer-based scaffolding was 0.46 (Belland, 
Walker, Kim, & Lefler, In Press). This is higher than the median effect size 
among meta-analyses of interventions in psychological research (g=0.324) (Cafri, 
Kromrey, & Brannick, 2010). It is also higher than the average effect size of edu- 
cational technology applications designed for mathematics education (ES=0.13) 
found in a recent review (Cheung & Slavin, 2013) and that of educational technol- 
ogy applications designed for reading instruction (ES=0.16) (Cheung & Slavin, 
2012). Computer-based scaffolding has been seen to have a very substantial effect 
size in prior research, as compared to that of similar interventions, and this warrants 
further research. 


2.4 Considerations as the Instructional Scaffolding 
Metaphor was Applied to Computer Tools 


The application of the instructional scaffolding metaphor to computer-based tools 
entails several new considerations, including the theoretical bases of computer- 
based scaffolding, how computer-based scaffolding should be designed, and the 
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interplay between computer-based and one-to-one scaffolding (Belland & Drake, 
2013; Belland, Gu, et al., 2013; Puntambekar & Hiibscher, 2005). As noted earlier, 
there are several traditions of computer-based scaffolding, each of which are based 
in different learning theory bases, including activity theory, ACT-R, and knowledge 
integration. This diversity of learning theory bases of scaffolding is not entirely un- 
expected, as Wood et al. (1976) never explicitly referenced learning theory in their 
seminal paper. The different theoretical bases inform how computer-based scaffold- 
ing is designed, what strategies it incorporates, and the role of the teacher in the 
support of student learning. 


2.4.1 Theoretical Bases of Computer-Based Scaffolding 


Instructional scaffolding was originally proposed to describe how teachers support- 
ed children as they learned to build with wooden blocks (Wood et al., 1976). What 
is often forgotten is that Wood et al. (1976) did not link scaffolding to a particular 
theoretical foundation. Rather, their paper was an attempt to describe how a tutor 
helped children put together wooden blocks to create shapes. Thus, while some 
theory figures into the paper, the authors did not describe the use of theory to design 
the scaffolding process. To the contrary, the description of the scaffolding process 
was grounded in observations of what actions the tutor took that led to student suc- 
cess. So in this way, the development of the scaffolding metaphor roughly followed 
the grounded theory approach (Glaser & Strauss, 1967). But, to help inform the 
design of scaffolding, later researchers attempted to link the construct to multiple 
theoretical bases. This plurality of underlying theoretical bases corresponds with 
different scaffolding approaches and different contexts in which scaffolding is used 
(Wood & Wood, 1996). 

Three primary theoretical bases of instructional scaffolding are activity theory, 
ACT-R, and knowledge integration. In this chapter, I describe these theoretical 
bases such that different approaches to scaffolding can be more easily understood. 


2.4.1.1 Activity Theory 


First, much scaffolding is linked to the social constructivism seen most prominently 
in the work of Vygotsky (1978), Leont’ev (1974), and Luria (1976). Commonly 
called activity theory, it likely made sense in the context of scaffolding in that Vy- 
gotsky famously based much of his work on the idea of a zone of proximal devel- 
opment—the set of tasks in which students could meaningfully participate with as- 
sistance (Smagorinsky, 1995; Vygotsky, 1978). Though it does not encompass all of 
Vygotsky’s work, and there are certainly many other important contributors to activ- 
ity theory, the critical underlying learning theory for scaffolding from this perspec- 
tive is cultural-historical activity theory (Belland & Drake, 2013; Pea, 2004)—a 
theory that was largely developed in the Soviet Union, in part due to an exhortation 
to apply the tenets of dialectical materialism to learning (Luria, 1979). 
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A central premise of cultural-historical activity theory is that the genesis of the 
development of new skills is in the external processes in which people engage (Ko- 
zulin, 1986; Leont’ev, 1974; Luria, 1976). This forms a sharp contrast with the 
assumptions of behaviorist theories of a stimulus-response origin of learning (Skin- 
ner, 1984), and that of information processing theories that learning occurs from the 
reception of new content and the subsequent use of encoding strategies such as mne- 
monics and rehearsal (Ausubel, 1980; Miller, 1956). According to an activity theory 
perspective, learning is not one’s reaction to the introduction of stimuli and associ- 
ated reinforcement and reinforcement removal or the use of rehearsal, mnemonics, 
and other cognitive strategies, but rather is the internalization of cultural and other 
knowledge inherent in external activity (D’ Andrade, 1981; Leont’ev, 2009; Luria, 
1976). The cultural knowledge can be embedded in such instructional support as 
computer-based scaffolding (Belland & Drake, 2013; Jonassen & Rohrer-Murphy, 
1999), or embedded in the support provided by and interactions with other individu- 
als (Engeström, 2009; Roth & Lee, 2007). 

According to activity theory, the external processes in which humans engage 
are shaped by a complex interaction between three entities—the individual, his/ 
her motives (goals), and signs (Leont’ev, 1974; Luria, 1979). From the perspective 
of an individual, a sign is the concept (signified concept) that another individual 
or object (signifier) represents (Barthes, 1994; Wertsch & Kazak, 2005). This rep- 
resentation can include what the individual thinks can be accomplished with the 
other individuals or objects, or what the object invites the individual to do. This 
perspective is informed by semiotics, which highlights the importance of individual 
perceptions when interacting with other individual and tools (Barthes, 1994). These 
individual perceptions can influence how individuals interact with other individuals 
and tools. From a semiotic perspective, each object has a signifier (form) and a sig- 
nified concept (what the object represents). For example, in the USA, the signifier 
of a stop sign is usually octagonal, red, and includes the writing “Stop.” However, 
the signified concept can vary among citizens. For some, it represents a suggestion 
to slow down. For others, it represents an order to stop and look both ways before 
proceeding through the intersection. Signs are arbitrary and are attached to enti- 
ties by groups or individuals on the basis of culture and history (Saffi, 2005). For 
example, there is nothing inherently sinister about clowns. Yet among many groups 
in Western cultures, clowns evoke a feeling of evil. This is due to the signification 
generated by the history (e.g., the serial killer John Wayne Gacy) and culture of the 
group. Society imposes or suggests classifications of objects (Barthes, 1994). How- 
ever, society does not impose the same classification to everyone because not all 
people experience the same society (Barthes, 1994). Classification of objects helps 
determine the meaning that signs will hold to individuals or groups. Individuals 
then interact with signs based on the signs’ meaning. 

Goals underlie all activity, and can be influenced by cultural and historical factors 
(Leont’ev, 2009). In this way, one would expect to see differences in approaches to 
actions between different cultures; indeed, such was found in the research of Luria 
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(1976) and Vygotsky (1978). Goals are crucial to the building of signs (Belland & 
Drake, 2013). It is important to recognize that goals are not always consciously 
identified and pursued (Locke & Latham, 2006). Nonetheless, such goals still form 
an important influence on the building of signs and, in turn, action (Saffi, 2005). 

As an example of how individuals’ cultures can shape their perception of a tool, 
consider language. Language can be a tool of symbolic violence, and the way in 
which it does or does not have the potential to be used in that way depends on one’s 
culture and, specifically, subculture (Bourdieu, 1982). One’s perception and use of 
language can then influence thought patterns. 

Thus, different individuals can build signs about tools and individuals in dif- 
ferent ways. This means then that they would perceive the tools and individuals as 
being useful to help accomplish different tasks. 


2.4.1.1.2 How New Skills Are Generated According to Activity Theory 


The use of tools and strategies can help learners gain cultural knowledge, as these 
reflect the core assumptions and ways of knowing of the target culture. Cultural 
knowledge can include constraints and guidance on how to categorize and count 
certain things (D’Andrade, 1981; Kozulin, 1986; Luria, 1976), symbol systems 
that frame how one views phenomena (Bourdieu, 1982; D’Andrade, 1981), and 
approaches to certain tasks (D’ Andrade, 1981; Luria, 1976). In this way, cultural 
patterns of interaction and ways of knowing are core to learning. 

From an activity theory perspective, the goal of instruction is to provide the tools 
and frameworks by which students can engage in the types of external actions that 
will allow them to internalize and integrate the desired content (Belland & Drake, 
2013; Jonassen & Rohrer-Murphy, 1999). Such tools and frameworks may embed 
representations of the cultural knowledge that one wishes to instill in students. By 
interacting with such tools and frameworks, individuals may have the opportunity 
to construct the target cultural knowledge. But this does not happen instantaneously; 
rather, it may be necessary to engage with several problems supported by the tools 
and frameworks to succeed in constructing the target cultural knowledge. It is also 
clear from activity theory that simply providing a set of tools and frameworks is not 
sufficient because individuals may interact with and use such differently based on 
their different experiences of culture and history (Belland & Drake, 2013; Leont’ev, 
2009; Luria, 1976). 


2.4.1.1.3 How Activity Theory Informs Instruction 


According to activity theory, productive interaction with tools and other individu- 
als in the process of solving authentic problems leads to learning (Leont’ev, 1974; 
Luria, 1976). It follows that instructional approaches aligned with activity theory 
stress the importance of collaboration and solving authentic problems (Jonassen & 
Rohrer-Murphy, 1999; Roth & Lee, 2007). Tools play a central role in instruction 
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informed by activity theory, but there is a recognition that the function of the tools 
provided to learners can vary, even when the physical form of the tools stays the 
same (Belland, 2010; Belland & Drake, 2013; Belland, Gu, Armbrust, & Cook, 
2015). 

An instructional approach grounded in activity theory takes a decidedly post- 
modern approach, in that it allows for multiple approaches and recognizes the im- 
portance of individual perspectives and those of members of the culture in which 
the student is operating (Friesen, 2012; Hlynka, 2012; Solomon, 2000). Further- 
more, such an approach would welcome the type of critique and dialogue that one 
would expect to see in a scientific laboratory or conference/publishing venue. Thus, 
such approaches would likely involve addressing a central, ill-structured problem 
(Jonassen, 2011; Jonassen & Rohrer-Murphy, 1999). Furthermore, students would 
be provided considerable latitude to address the problem in the manner that best 
suited them. 


2.4.1.1.4 How Activity Theory Informs Scaffolding 


Activity theory can describe the social mediation process of scaffolding (Engeström, 
2009; Jonassen & Rohrer-Murphy, 1999; Roth, 2012). Goals can influence how 
learners interpret and use scaffolds (Belland & Drake, 2013; Belland, Glazewski, & 
Richardson, 2011). Specifically, when learners view scaffolds, they do not all see 
the same thing; rather, they build a sign based on goals and cultural and historical 
factors (Belland & Drake, 2013; Leont’ev, 1974; Wertsch, 1991). A sign refers to 
the learners’ internal representation of what the tool is, what it should be used for, 
and what can be accomplished with it (Belland & Drake, 2013; Wertsch, 1991). 
Learners build signs on the basis of culture and history—one’s individual histo- 
ry with similar tools and the situations in which they are used (Belland & Drake, 
2013). Furthermore, due to the influence of culture and history on their definition, 
signs are not the same for all individuals, since by definition each individual will 
experience different cultural influences and histories (Barthes, 1994; Saffi, 2005). 
When students interact with the scaffold, they interact with the sign (i.e., signified 
concept) rather than with a static, unchanging tool (Belland & Drake, 2013). This 
means that different learners can see and use scaffolds in different ways (Belland, 
2010; Belland & Drake, 2013; Belland et al., 2011). Thus, when designing scaffold- 
ing, it is important to think about the processes and situations in which the scaffold- 
ing will be used (Akhras & Self, 2002; Belland & Drake, 2013). 

Activity theory explains that tools such as scaffolding do not merely transmit 
human action from one forum to another, as an ax transmits the force produced by 
swinging one’s arms to the surface area of the blade. Rather, as a psychological tool, 
scaffolding transforms and extends human action first in external action, and then 
that same transformed external action can be internalized (Belland & Drake, 2013; 
Kozulin, 1986). In this way, the cultural knowledge inherent in the scaffold can be 
internalized in the learner. Cultural knowledge can be defined as knowledge, ten- 
dencies, and skills that are shared by a group of people (Hogan & Maglienti, 2001; 
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Leont’ev, 1974; Luria, 1976). Cultures in this case refer not only to national cul- 
tures like German or Indonesian, but can include members of an occupation (e.g., 
civil engineers, bankers) or of a particular interest group (e.g., bird watchers, coin 
collectors). For example, the cultural knowledge of civil engineers may include 
methods to elicit and prioritize client needs when discussing a project. The cultural 
knowledge of bird watchers may include strategies to quickly distinguish between 
the calls of different species of birds. Cultural knowledge is often implicit, in that 
members are not always consciously aware of it. To succeed at thinking or acting 
like a member of a particular culture, it is important to take into account cultural 
knowledge and incorporate such into support (e.g., scaffolding). 

In short, scaffolding informed by cultural-historical activity theory seeks to help 
learners use cultural tools as they engage in higher-order tasks, and assimilate such 
into their own practice (Belland & Drake, 2013; Jonassen & Rohrer-Murphy, 1999). 
This in turn helps students develop higher-order psychological processes (Vygotsky, 
1962). Thus, from an activity theory perspective, when designing scaffolding, it is 
important to think broadly about the dispositions and modes of thinking that one 
wishes to develop in students, rather than about discrete skills students need to 
develop (Akhras & Self, 2002; Belland & Drake, 2013). This may be accomplished 
through the use of ethnographies of the professions of interest. This can allow de- 
signers to find out the key dispositions and thinking strategies employed by mem- 
bers of the profession and then think about how such can be applied in problems that 
are accessible to the student population. 


2.4.1.2 ACT-R 


Much research views scaffolding as a vehicle to promote student learning of higher- 
order skills through the creation and optimization of production rules and learning 
of declarative knowledge (VanLehn, 2011). Such production rules can then be used 
in sequence to produce the target higher-order skill. This view of scaffolding draws 
on the Adaptive Character of Thought-Rational (ACT-R) learning theory (Ander- 
son, 1996). 


2.4.1.2.1 Theoretical Background 


In cognitive science, there has long been a push to develop a unitary theory of 
cognition (Laird, 2008; Newell, 1973). According to this idea, rather than develop 
many specialized theories and conduct various investigations about different cog- 
nitive phenomena, cognitive scientists and psychologists should strive to develop 
and test a theory by which all human cognition can be explained. If true, such a 
theory would show that all human cognition is the product of the application of dif- 
fering combinations of the same subskills (Anderson, 1983, 1990). According to a 
unitary theory of cognition, there is nothing special about any cognition—that any 
thought, be it a breakthrough or simply a determination of what to eat for lunch, is 
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an assemblage of various components of the same set of declarative knowledge and 
production rules (Anderson, 1983; Laird, 2008). Within this context, John Anderson 
and colleagues developed a series of learning theories—the ACT series of theories 
of cognition—and attempted to posit these as unitary theories of cognition (Ander- 
son, 1983, 1990). Anderson and colleagues have worked on the development and 
testing of intelligent tutoring systems in part to test and refine the tenets of ACT 
theories of cognition (Anderson et al., 1997; Koedinger & Corbett, 2006). 

ACT-R is a recent version of the ACT series of theories of cognition. Lying 
behind ACT-R is a theory of rational action, of which a critical assumption is that 
people will always consciously choose to act in the manner that they perceive best 
serves their own interests (Anderson, 1990). This draws on research related to the 
theory of reasoned action, according to which, in the aggregate, people act in accor- 
dance with salient personal beliefs (Ajzen, 1991). This does not imply a conscious 
decision to act in accordance with the salient belief before each action (Ajzen, 1991; 
Anderson, 1990). For example, personal beliefs about the efficacy of a particular 
strategy can predict one’s attitudes about the strategy and, in turn, propensity to use 
such strategy in a salient situation (Ajzen, 1991). 

In the context of ACT-R, it is important to note that rational action implies that 
there are goals inherent in cognitive systems (Anderson, 1990). Such goals are spec- 
ified in the way that a problem is framed (Anderson, 1990). Once such goals are 
identified, individuals determine the most appropriate production rules and declara- 
tive knowledge to deploy to achieve the goals (Anderson, 1990). Such decisions are 
informed with reference to utility values that were generated in the creation of the 
production rule and thus associated with the latter. When individuals are confronted 
with a new problem, they search through the available production rules, and pick 
the one that has the highest utility value for the situation (Anderson et al., 2004). 

According to ACT-R, a cognitive theory needs only concern itself with three lev- 
els of analysis: the biological level, the algorithmic level, and the rational level (An- 
derson, 1990). The biological level is what resides in the head. One cannot model 
it exactly, but one can approximate it through the use of an implementation model. 
The algorithmic level is the set of procedures and strategies by which informa- 
tion can be encoded, retrieved, and deployed in problem-solving (Anderson, 1987, 
1990). The rational level concerns the constraints to which cognition needs to ad- 
here to be rational, defined as working towards the agent’s goals (Anderson, 1990). 


2.4.1.2.2 How New Skills Are Generated According to ACT-R 


Using ACT-R, complex skills can be broken down into knowledge chunks and pro- 
duction rules, which dictate how to apply the knowledge to solve problems (An- 
derson, 1996). Knowledge chunks all encode two or more elements, and how they 
relate (Anderson, 1983). Chunks never exceed seven elements, as informed by the 
cognitive information processing theory finding that one can at most manipulate 
6-8 pieces of information in short-term memory at a time (Miller, 1956). For exam- 
ple, chunks can include (relation: love, agent: baby, object: pacifier), (relation: hate, 
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agent: baby, object: dirty diaper), and (relation: hate, agent: baby, object: hunger). 
According to ACT-R, one cannot directly teach production rules. Rather, one needs 
to teach the knowledge associated with the production rule in declarative form and 
invite the learner to practice applying the knowledge in the context of problems. In 
other words, all knowledge begins in declarative form, and can become procedural 
when students have applied it enough to authentic problems (Anderson, 1983; An- 
derson et al., 2004). When students first learn knowledge chunks and attempt to 
apply such, they do so using general procedures (Anderson, 1983). This process 
requires the students’ active interpretation. For example, new parents might apply 
the knowledge chunks (relation: love, agent: baby, object: pacifier), (relation: hate, 
agent: baby, object: dirty diaper), and (relation: hate, agent: baby, object: hunger) 
in succession when their baby cries. Desperate to console the baby, they attempt to 
interpret what the baby wants by applying the chunks using the general framework 
that when someone is unhappy, it is important to figure out the root of the unhap- 
piness and that one can do so through the process of elimination. As they apply the 
new knowledge enough using general procedures, they begin to develop production 
rules—strategies that they can employ without the use of active interpretation. In 
other words, the student knows that in X situation, one can apply knowledge chunk 
Y using strategy Z, and can apply strategy Z in X situation without actively inter- 
preting the situation (Anderson, 1983). People are not always consciously aware 
of production rules, but not being consciously aware of production rules does not 
prevent their application (Anderson et al., 1997). 

ACTER posits that learning complex skills involves learning the right declarative 
knowledge chunks and generating the right production rules in the right order as 
well as practicing deploying the knowledge chunks by way of production rules in 
the context of solving problems (Anderson, 1996; Anderson et al., 1997). ACT-R 
also sees an additional knowledge set brought to bear when solving a problem—the 
goal module—which governs what individuals aim to do when presented stimulus 
materials that could prompt multiple actions (Anderson et al., 2004). ACT-R also 
sees excessive failure as not conducive to learning and thus advocates maximizing 
successful practice and minimizing opportunities for excessive failure (Koedinger 
& Aleven, 2007). Ultimately, the goal of ACT-R is that students practice applying 
content knowledge to problems and, in the process, generate and optimize produc- 
tion rules that govern the application of such declarative knowledge to problems 
(Aleven, Stahl, Schworm, Fischer, & Wallace, 2003). 


2.4.1.2.3 How ACT-R Informs Instruction 


The goal of instruction according to ACT-R is to present the right knowledge chunks 
to students in the right order and provide opportunities for structured practice ap- 
plying the knowledge chunks in the context of problem-solving (Anderson et al., 
1997). Instruction should also minimize the chances for failure and maximize the 
chances of success (Anderson, 1996; Koedinger & Aleven, 2007). Along this vein, 
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prior to beginning the design of instruction, designers should determine what is to 
be learned and how (Baker et al., 2007). The material to be learned includes declara- 
tive knowledge and production rules by which the declarative knowledge can be 
applied to problems. But the declarative knowledge is to be transmitted to students, 
and scaffolding should help students engage in the type of problem-solving practice 
by which they can generate production rules. Unlike with activity theory, there is 
usually no premium placed on collaboration, although it should be noted that some 
intelligent tutoring systems are designed to support collaboration (Diziol, Walker, 
Rummel, & Koedinger, 2010). Furthermore, there is no need necessarily for an 
overall problem around which all learning is centered; rather, an intelligent tutoring 
system may incorporate a sequence of related problems. 

Taking a step back from the specifics of ACT-R, one may note that underneath 
the theory is a positivist mindset: that the reality is out there and known, and instruc- 
tion should transmit to students what is known about reality. This is true to a certain 
extent. However, in ACT-R, students generate production rules, and such produc- 
tion rules may not be exactly the same amongst all students. It is important to not 
fall into the trap of thinking that all positivist traditions are simplistic and harmful 
to learning; rather, positivist approaches can form a solid cornerstone in science, 
technology, engineering and mathematics (STEM) education (Matthews, 2004). 


2.4.1.2.4 How ACT-R Informs Scaffolding 


One of the tenets of ACT-R that most influences the design of scaffolding is the idea 
that it is best to maximize successful practice and minimize unsuccessful practice. In 
this way, the exact amount of scaffolding informed by ACT-R often can be modified 
based on (a) model tracing of students’ abilities according to their progress through 
the systems and success or lack thereof on tasks, and (b) student self-selection of 
hints (Koedinger & Aleven, 2007). Adjustment based on model tracing attempts 
to automatically increase or decrease base student support based on the system’s 
estimation of students’ current abilities. Adjustment based on self-selection most 
often involves the provision of hints on next steps or strategies to solve the target 
problem. Most often, the first time a student requests a hint, the provided hint helps 
a little, the next hint requested helps even more, and the third hint requested is the 
bottom-out hint—it tells students what to do (Koedinger & Aleven, 2007). 

Next, given that ACT-R focuses on promoting the learning of smaller produc- 
tion rules that govern the application of declarative knowledge chunks, scaffolding 
in ACT-R is at a fairly small grain size, especially in comparison with scaffolding 
informed by activity theory (Anderson, 1983; Belland & Drake, 2013). That is, 
scaffolding focuses on subprocesses that contribute to solving problems, rather than 
macro-processes. In this way, scaffolding informed by ACT-R leads students step- 
by-step through a series of sub-strategies that are said to lead to success at solving 
the target problem (Anderson et al., 1997). 
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2.4.1.3 Knowledge Integration 


There is also much scaffolding that is developed to lead to the type of deep content 
learning that Marcia Linn called knowledge integration (Linn, 2000). Deep content 
learning means more than simply being able to recall information, but rather being 
able to describe it in one’s own words and apply it in novel situations (Belland, 
French, & Ertmer, 2009; Bloom, Englehart, Furst, Hill, & Krathwohl, 1956). Such 
application in novel situations may happen when individuals attempt to create a 
model of the new problem; in so doing, they may make reference to their current 
mental models (Kolodner, 1993; Nersessian, 2008). Having an integrated mental 
model to which to refer improves reasoning efficiency and the likelihood of success- 
ful reasoning (Ifenthaler & Seel, 2013; Johnson-Laird, 2001). Knowledge integra- 
tion is evidenced by integrated mental models describing how nature works, and the 
knowledge that the same principles of how nature works apply equally well inside 
and outside of school (Clark & Linn, 2013; Kali, Orion, & Eylon, 2003; Linn, 2000). 
Furthermore, students who evidence knowledge integration should be able to apply 
their integrated mental models to novel problems (Linn, 2000; Linn et al., 2003). 


2.4.1.3.1 Theoretical Background 


The knowledge integration framework was built off of the knowledge in pieces 
theory (diSessa, 1988), the anchored instruction framework (The Cognition and 
Technology Group at Vanderbilt, 1990), situated learning in collaborative groups 
(Brown & Campione, 1994; Lave & Wenger, 1991), and research that suggests that 
learning outcomes in science instruction would be best served when one focuses on 
a smaller number of core concepts (Bierman, Massey, & Manduca, 2006; Eylon & 
Linn, 1988). These perspectives are explained in the paragraphs that follow. 
According to the knowledge in pieces theory, students come to school having 
developed intuitive theories of how physical objects behave under particular cir- 
cumstances; some of these mini-theories come close to describing phenomena of 
interest accurately, while others are farther away from describing said phenomena 
accurately (diSessa, 1988; Taber, 2008). Such mini-theories are not developed as 
most theories are—through careful reflection on a variety of observations in light 
of other research and theories. Rather, they are “abstractions from common experi- 
ences”—such as the idea that force can move objects (diSessa, 1988, p. 3). These 
mini-theories do not together constitute a larger, more comprehensive theory. Fur- 
thermore, students do not have the right pieces of knowledge to together explain 
how physical objects behave in a scientifically accurate way. Some research has 
suggested that such incomplete mini-theories do not necessarily prompt the teach- 
ing of correct information to replace the existing mini-theories (Spada, 1994). Rath- 
er, instruction needs to help fill in the gaps in students’ knowledge (diSessa, 1988). 
In anchored instruction, students’ learning is centered in an authentic problem 
situation, which prompts students to define and pursue learning issues (Bransford, 
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Plants, & Vye, 2003; The Cognition and Technology Group at Vanderbilt, 1990). 
It was designed to prevent the problem of inert knowledge—knowledge that 
individuals know and can activate when asked to, but they do not spontaneously 
do so even when a presented problem warrants it (The Cognition and Technology 
Group at Vanderbilt, 1990). Anchored instruction seeks to promote broad transfer 
(Bottge, Rueda, Kwon, Grant, & LaRoque, 2007; The Cognition and Technology 
Group at Vanderbilt, 1990). Within anchored instruction, student learning is cen- 
tered around several challenges, defined as mini problems that students need to 
address. Students are also given tools and information with which the challenges 
can be addressed. Typically, all information and tools that are needed to address the 
challenges are contained within the anchored instruction program (The Cognition 
and Technology Group at Vanderbilt, 1990). Students are encouraged to revisit chal- 
lenges after they gather feedback from peers and teachers (Bransford et al., 2003; 
The Cognition and Technology Group at Vanderbilt, 1990). 

One of the key tenets of the situated learning theory is that all learning takes 
place in a context, and that to maximize the potential applicability of new learn- 
ing, one should ensure that the learning context is similar to the context in which 
the new content is to be applied (Clancey, 2008; Lave & Wenger, 1991). By first 
observing and then participating at the edges of authentic work groups, students 
can gradually engage in legitimate peripheral participation, whereby they can gain 
the skills necessary to participate fully in the community of practice (Collins et al., 
1989; Herrington & Oliver, 2000; Lave & Wenger, 1991). This is important such 
that learners have the contextual cues to access the schemas they create (Greeno & 
van de Sande, 2007; Lave & Wenger, 1991). 

Much research indicates that science learning outcomes are maximized when 
science curricula covers a smaller number of concepts at a deep level (Achieve, 
2013; Clark, 2000; Duschl, 2008; National Research Council, 2007; Pritchard, Bar- 
rantes, & Belland, 2009). Specific learning outcomes to be developed include an 
understanding of science at a conceptual level (as opposed to a set of declarative 
facts) (Pritchard et al., 2009), learning of concepts and principles that apply across 
a variety of STEM fields (Achieve, 2013; National Research Council, 2011), and 
higher-order thinking skills such as problem-solving (Abd-El-Khalick et al., 2004; 
Jonassen, 2000, 2011) and argumentation abilities (Belland, Glazewski, & Richard- 
son, 2008; Ford, 2012; Kuhn, 2010). 


2.4.1.3.2 How New Skills Are Generated According to Knowledge Integration 


According to the knowledge integration framework, educators should endeavor to 
help students develop integrated mental models with which they can view scientific 
phenomena (Linn, 2000; Linn et al., 2003). Students come to science class with 
certain preconceptions about how nature works. Instruction then should not attempt 
to replace such knowledge, but help students integrate new knowledge about the 
natural world into their existing mental models (Linn, 2000; Linn et al., 2003). Stu- 
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dents should also be guided to and have the opportunity to make sense of multiple, 
conflicting observations (Clark & Linn, 2013). In this process, they can distinguish 
among and re-order their preexisting ideas and new ideas that are generated (Clark 
& Linn, 2013). This can be done when students address a multitude of problems in 
context, aided by context-specific support (e.g., scaffolding) (Clark & Linn, 2013; 
Kali & Linn, 2008). In so doing, it is important that students see a variety of cases 
that conflict with their preexisting ideas related to the topic at hand (Linn, 2000). 
When they attempt to make sense of how the new cases conflict with their preexist- 
ing ideas, they have the potential to move toward knowledge integration (Clark & 
Linn, 2013; Linn et al., 2003). 


2.4.1.3.3 How Knowledge Integration Informs Instruction 


A central premise of knowledge integration is that students make observations of 
the world in a variety of settings, and attempt to use these observations to generate 
mental models with which they can explain natural phenomena (Linn, 2000). But 
they struggle to sort out these often conflicting observations without detailed and 
structured instructional guidance (Kali et al., 2003; Linn et al., 2003). Students who 
believe that science is an unchanging body of knowledge struggle especially hard 
to develop integrated knowledge about science (Songer & Linn, 1991). Instruction 
following the knowledge integration approach includes the following processes: 
Invitation to articulate existing ideas, provision of normative ideas, invitation to 
distinguish among preexisting and normative ideas, and invitation to reflect on what 
was learned (Clark & Linn, 2013). Compared to instruction informed by activity 
theory, knowledge integration aims for a more highly structured instructional ap- 
proach (Clark & Linn, 2013; Jonassen & Rohrer-Murphy, 1999; Kali et al., 2003). 

Knowledge integration is positivistic to the extent that designers are said to be 
able to identify the ultimate truth, which then can be communicated to students 
(Clark & Linn, 2013; Linn, 2000; Matthews, 2004). However, students’ preexist- 
ing ideas about natural phenomena are treated as valuable pieces of a future mental 
model, and this is more postmodern (Hlynka, 2012; Solomon, 2000). 


2.4.1.3.4 How Knowledge Integration Informs Scaffolding 


According to the knowledge integration framework, scaffolding is important inso- 
far as it helps enhance students’ mental models of scientific concepts, integrating 
new content with their preexisting knowledge (Linn et al., 2003). To do this, it is 
important to elicit prior science ideas from students, help them gain new ideas while 
addressing problems, and help them to see where the new ideas fit with their preex- 
isting ideas (Chang & Linn, 2013; Clark & Linn, 2013). To promote knowledge in- 
tegration, it is important to make science accessible, make thinking visible, provide 
social supports, and promote autonomy (Linn, 2000). One can do this by inviting 
students to articulate their ideas, providing collaboration tools, providing all of the 
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information and tools students need to solve the problem within the system, and 
inviting students to reflect on what they have learned. 


2.4.1.4 Comparison of Theoretical Foundations 
2.4.1.4.1 Assumptions About Learning 


First, one notes that each of these theoretical bases have starkly different assump- 
tions about learning. One such difference is in their answers to a persistent philo- 
sophical question in education: to what extent should educators define what is to be 
learned? According to ACT-R (Akhras & Self, 2002; Anderson, 1996) and knowl- 
edge integration (Linn, 2000), educators should determine what is to be learned 
and how learning experiences might be arranged to lead to such learning. While 
in activity theory there is not the thought that any learning is good learning, still 
there is not as much of a focus on educators unilaterally determining learning goals 
and scripting learning activities to inexorably lead to such learning goals (Jonas- 
sen & Rohrer-Murphy, 1999; Kozulin, 1986; Leont’ev, 1974). Rather, through their 
interaction with other individuals and tools, supported by scaffolding, learners de- 
velop needed skills. The exact skills that are picked up can vary by learners, their 
goals, the culture in which they operate, and so forth (Akhras & Self, 2002; Belland 
& Drake, 2013; Jonassen & Rohrer-Murphy, 1999). This has major implications 
for the design of scaffolding. When designing intelligent tutoring systems based in 
ACT-R, one needs to model the knowledge structures that are thought to undergird 
the target higher-order skill (Aleven et al., 2003; Baker et al., 2007). Similarly, 
when designing scaffolding grounded in knowledge integration, one needs to model 
the knowledge structures inherent in the type of sophisticated mental model one is 
targeting (Kali & Linn, 2008; Linn et al., 2003). However, when designing scaffold- 
ing from an activity theory perspective, one needs to model the process by which 
students would engage with an ill-structured problem, including their individual 
mental process and how they interact with others (Akhras & Self, 2002; Belland & 
Drake, 2013). Only then could one consider what type of scaffolding tools could be 
useful for students in the learning context (Akhras & Self, 2002; Belland & Drake, 
2013). 

Next, the theoretical bases differ in terms of their view on the granularity of 
knowledge, or lack thereof. One of ACT-R’s central premises is that any skill can 
be broken down into subskills that can be taught in succession in order to teach 
the overall skill (Anderson, 1990). This perspective is different from that of activ- 
ity theory, which views overall skills in a holistic manner, and sees a need to help 
students develop such skills in their entirety in the context of addressing authentic 
problems, supported by tools and other individuals (Leont’ev, 2009; Luria, 1976). 
Comparing knowledge integration with activity theory and ACT-R on granularity 
of knowledge is not the most productive comparison, as the former and the latter 
models seek to promote different learning outcomes: integrated mental models ver- 
sus higher-order thinking skills. 
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2.4.1.4.2 Goals of Scaffolding 


The goals of scaffolding informed by each of these theory bases are influenced by 
the assumptions of the latter. To help with the explanation of the different theoretical 
bases, consider the goal of teaching problem-solving skill A. According to activity 
theory, the goal of instruction is to help learners gain higher-order skills in interaction 
with others (Leont’ev, 1974, 2009; Roth & Lee, 2007). Thus, instruction should give 
learners the opportunity to use problem-solving skill A when interacting with other 
individuals and tools. According to this perspective, a skill such as problem-solving 
skill A cannot be reduced to smaller components. Thus, students need to meaning- 
fully participate in the performance of the whole skill. Scaffolding can extend learn- 
ers’ skill sets as they engage in the target task in collaboration with other individu- 
als. From an activity theory perspective, scaffolding is a tool with which students 
can engage in collaborative problem-solving, and, by extension, generate the target, 
higher-order skill (e.g., argumentation ability; Belland & Drake, 2013; Jonassen & 
Rohrer-Murphy, 1999). Such scaffolding can promote the enhancement of students’ 
problem-solving abilities (Ge & Land, 2004; Raes, Schellens, De Wever, & Vander- 
hoven, 2012), argumentation abilities (Aufschnaiter, Erduran, Osborne, & Simon, 
2008; Belland et al., 2008; Jeong & Joung, 2007) as well as abilities to apply disci- 
pline-specific strategies. It focuses on student goals while engaging in the underlying 
problem-solving activity and attempts to be in the form that students could perceive 
of as useful when engaging with the problem. From an activity theory perspective, 
scaffolding need not be designed to minimize the amount of failure, as it recognizes 
failure as an event that can promote learning (Reiser, 2004; Simons & Ertmer, 2006). 

On the contrary, from an ACT-R perspective, instruction should transmit declara- 
tive knowledge that students can practice applying when solving problems; in so 
doing, students generate production rules, which guide how to perform smaller sub- 
skills in sequence to perform the entire target skill (Anderson, 1996). Continuing 
with the example, scaffold developers working from an ACT-R perspective would 
think about how to break down problem-solving strategy A into smaller subskills 
(Baker et al., 2007). Declarative knowledge needed to engage in the subskills would 
be identified, and would be programmed to be delivered to learners in sequence. 
Scaffolding would be set up to help learners to apply the declarative knowledge in 
the context of smaller problems and develop production rules in the process. The 
idea is that the learner would be able to string together the generated production 
rules to perform problem-solving strategy A. So scaffolding informed by ACT-R 
has a smaller grain size: it is designed to help students get the practice they need to 
generate production rules for declarative knowledge that is the focus of the instruc- 
tion (Koedinger & Corbett, 2006; Means & Gott, 1988). Such scaffolding would 
provide the opportunity for students to have successful practice applying the knowl- 
edge that the intelligent tutoring system delivered. It would also be designed to 
minimize failure through the use of multiple methods to determine whether adding 
or removing scaffolding is necessary, including self-selection of hints and the use 
of model tracing of students’ abilities to inform adding and removing scaffolding 
(Koedinger & Aleven, 2007; Koedinger & Corbett, 2006). 
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From a knowledge integration perspective, the goal of instruction is to help 
learners’ existing mental models evolve to reflect more generally accepted scien- 
tific theories and perspectives (Davis & Linn, 2000; Linn et al., 2003). The idea is 
that with more sophisticated mental models, learners would be able to effectively 
address new problems, an idea with strong support in educational research (Gentner 
& Stevens, 2014; Ifenthaler & Seel, 2013; Vosniadou, Ioannides, Dimitrakopoulou, 
& Papademetriou, 2001). At the same time, knowledge integration does not seek to 
replace learners’ existing conceptions (Linn et al., 2003). Scaffolding designed from 
a knowledge integration perspective would elicit preexisting knowledge related to 
the new content to be learned, present new content, and help students to integrate 
the new content with preexisting knowledge while engaging with a problem (Chang 
& Linn, 2013; Clark & Linn, 2013). 

Strategies deployed by scaffolding centered on each of these theory bases would 
vary as well. Scaffolding grounded in activity theory would tend to incorporate 
strategies that are highly valued in the target culture, given the importance of cul- 
tural knowledge from an activity theory perspective (Engeström, 2009; Leont’ev, 
2009; Luria, 1976). Such strategies would not necessarily be designed to produce 
student success in the fastest manner possible, but to promote meaningful engage- 
ment in the problem (Belland & Drake, 2013). Scaffolding grounded in ACT-R 
would tend to be designed to promote student success as quickly as possible, as 
ACTER posits struggle as an impediment to learning (Anderson et al., 1997; Self, 
1998). Scaffolding designed from a knowledge integration perspective would aim 
to activate prior knowledge and promote the acquisition of new knowledge and the 
integration of new knowledge with existing knowledge (Clark & Linn, 2013; Davis 
& Linn, 2000). 


2.4.1.4.3  Operationalization of Scaffolding 


As the goals of scaffolding differ depending on the theoretical framework that un- 
dergirds their design and use, so does the operationalization of scaffolding. From an 
activity theory perspective, stretching students’ abilities to the maximum potential 
is desired (Jonassen & Rohrer-Murphy, 1999; Roth & Lee, 2007). As such, one 
designs scaffolding so as to maximize productive struggle (Belland, 2014; Reiser, 
2004; Simons & Ertmer, 2006). Productive struggle refers to struggle within the 
areas of the task that are most likely to lead to target learning outcomes and which 
is not likely to lead to disengagement (Belland, Kim, et al., 2013). Thus, within 
reason, struggling is not cause for concern, but rather represents an opportunity for 
learning. In this way, adding scaffolding is not desirable, but rather removing (fad- 
ing) scaffolding is (Pea, 2004). 

From an ACT-R perspective, struggle is counterproductive, and thus intelligent 
tutoring systems allow students to request hints when they struggle (Anderson et al., 
1997; Koedinger & Aleven, 2007). The first hint is more subtle, but as the student 
requests more, the hints become more direct, eventually ending in a bottom-out hint 
that provides the answer (Koedinger & Aleven, 2007; Koedinger & Corbett, 2006). 
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Thus, intelligent tutoring systems leave less latitude for choice in problem-solving 
direction and action than does scaffolding informed by activity theory. 

From a knowledge integration perspective, the goal of scaffolding is to help stu- 
dents fill in gaps in their existing mini-theories about how nature works (Clark & 
Linn, 2013; Linn, 2000). As such, the promotion of productive struggle is not par- 
ticularly important. But, at the same time, struggle is not something to be avoided 
at all costs. 


2.4.1.4.4 The Role of the Teacher 


Theory and empirical evidence indicates that computer-based scaffolding informed 
by activity theory and knowledge integration does not work without the provision 
of one-to-one scaffolding by teachers (Davis & Linn, 2000; McNeill & Krajcik, 
2009; Muukkonen et al., 2005; Puntambekar & Kolodner, 2005; Saye & Brush, 
2002). Through one-to-one scaffolding, teachers do things like press for under- 
standing and question student understanding, actions for which human teachers are 
much more suitable than computer-based scaffolding, or at least computer-based 
scaffolding as informed by activity theory and knowledge integration (Middleton & 
Midgley, 2002; Pressley, Gaskins, Solic, & Collins, 2006). Meanwhile, computer- 
based scaffolding can help with tasks for which automated computer tools are better 
suited, such as persistent support related to important concepts and strategies that 
figure into the problem solution (Belland, Gu, et al., 2013; Muukkonen et al., 2005; 
Saye & Brush, 2002). 

Often, intelligent tutoring systems are meant to be largely self-contained learn- 
ing systems, in which computer-based scaffolding engages in some of the question- 
ing of student understanding that is otherwise reserved for human teachers (Koed- 
inger & Aleven, 2007; Koedinger & Corbett, 2006). In these cases, teachers are still 
important, but their role is more as someone to help smaller number of students who 
continue to struggle even while using the intelligent tutoring system (Diziol et al., 
2010; Koedinger & Corbett, 2006). However, there are intelligent tutoring systems 
that posit a more active role for the teacher in suggesting the types of help that stu- 
dents seek and planning instruction (Ainsworth, Grimshaw, & Underwood, 1999; 
Dimitrova & Dicheva, 1998). 


2.4.2 Design of Computer-Based Scaffolding 


Computer-based scaffolding needs to be designed and developed before target stu- 
dents use it (Belland, 2014). At a global level, this design process can involve thor- 
oughly understanding the process/skill to be promoted (Murray, 1999; Quintana, 
Krajcik, & Soloway, 2003), predicting the difficulties that target students will face 
in the task (Baker et al., 2007; Quintana et al., 2003), determining smaller subskills 
that are involved in the target skill (Koedinger & Aleven, 2007), considering the 
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situations in which the tool will be used (Akhras & Self, 2002; Belland & Drake, 
2013), and designing strategies to help students overcome difficulties to assume 
expertise on the underlying process/skill (Quintana et al., 2003). For example, a 
scaffold designer may need to carefully define what it means to be an expert related 
to a particular task and define the gap in expertise between experts and the target 
learners (Baker et al., 2007; Murray, 1999; Quintana et al., 2003). As part of this 
process, it is important to determine which elements of the gap are the most difficult 
for students to overcome. One can do this through difficulty factors analysis—an 
empirical technique in which the designer varies different task elements in an effort 
to determine which is the most difficult (Baker et al., 2007). Designers also need to 
consider the information, activity, management, and reflection needs that learners 
will face when engaging in the target activity (Quintana et al., 2003). It is important 
to think about not only the strategies that will be embedded in the scaffolding soft- 
ware, but also about the physical manifestation of these strategies (Quintana et al., 
2003). One also needs to consider the types of situations in which learners will use 
the proposed scaffold—with whom they interact, what they do, and what needs they 
face (Akhras & Self, 2002; Belland & Drake, 2013). 

The design process can vary based on the underlying type/tradition of scaffold- 
ing (e.g., scaffolding embedded in intelligent tutoring systems, computer-based 
scaffolds to support knowledge integration). For example, in the first stage of the 
design of intelligent tutoring systems, many designers classify target skills in terms 
of production rules and declarative knowledge (Baker et al., 2007; Koedinger & 
Corbett, 2006; Murray, 1999). In the initial stages of designing scaffolding to support 
knowledge integration, defining the content to be learned, and how it might be most 
productively organized in a mental model, is key (Clark & Linn, 2013; Linn et al., 
2003). Furthermore, it is important to consider the existing knowledge target learners 
will bring to the learning task (Linn et al., 2003). For scaffolding designed according 
to the activity theory perspective, it is important to characterize the target skill in a 
holistic manner and consider the types of situations in which students can gain the 
skill and what support would be needed to enable productive interaction with others 
in the completion of the task (Akhras & Self, 2002; Belland & Drake, 2013). It is also 
important to consider the cultural knowledge required to perform the target skill sat- 


isfactorily, and how such knowledge can be embedded in the scaffold (Luria, 1976). 


2.4.3 Interplay Between Computer-Based and One-to-One 
Scaffolding 


A recent review indicated that technology-based educational innovations are rarely 
successful unless participating teachers engaged in a sustained professional devel- 
opment program for at least 1 year (Gerard, Varma, Corliss, & Linn, 2011). The 
reason for this is that with less professional development, teachers are likely to 
spend most of their time addressing technical problems, and little time helping their 
students engage in high-level thinking (Gerard et al., 2011). In this way, students do 
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not have the opportunity to benefit from one-to-one scaffolding from their teachers 
and their learning and performance suffers (Gillies & Boyle, 2006; Maloch, 2002; 
Raphael, Pressley, & Mohan, 2008). 

As noted previously, one-to-one scaffolding and computer-based scaffolding each 
have unique strengths (see Table 2.1 on page 24). One-to-one scaffolding is the most 
dynamic form of scaffolding (Chi, 1996; van de Pol et al., 2010; Wood, 2003), more 
dynamic even than scaffolding in intelligent tutoring systems (Koedinger & Aleven, 
2007). One-to-one scaffolding is particularly good at pressing students for under- 
standing and prompting high-level performances (Levpušček, Zupančič, & Sočan, 
2013; Middleton & Midgley, 2002; Pressley et al., 2006; Turner et al., 1998). But 
one-to-one scaffolding is limited in terms of scale and availability in that it requires 
one teacher to work on a one-to-one basis with one student, a luxury in most K-12 and 
other classrooms (Belland, 2014; Rodgers, 2004; van de Pol et al., 2010). Because 
teachers cannot work one-to-one with all students in their class at the same time, it 
is important to also provide computer-based scaffolding to share the scaffolding load 
(Belland, 2014; Belland, Gu, et al., 2013; Saye & Brush, 2002). Computer-based 
scaffolding is available all the time to all students. It also has infinite patience, which 
can occasionally be an issue with one-to-one scaffolding. By thoroughly designing 
computer-based scaffolding ahead of students’ engagement in learning activities, one 
can also avoid the possibility of scaffolding messages being provided in qualitatively 
different ways to different student subgroups (Mertzman, 2008). 

One-to-one scaffolding can make computer-based scaffolding more effective by 
reinforcing themes and pressing students to (a) consider the central problem and the 
learning material critically, and (b) question their own understanding (Belland, Gu, 
et al., 2013; Gerard et al., 2011; McNeill & Krajcik, 2009; Muukkonen et al., 2005; 
Puntambekar & Kolodner, 2005). The synergy afforded by pairing strong computer- 
based scaffolding with effective one-to-one scaffolding can promote high levels 
of achievement among students (Belland, Burdo, et al., 2015; McNeill & Krajcik, 
2009; Tabak, 2004). 
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Chapter 3 
Context of Use of Computer-Based Scaffolding 


Abstract The contexts in which computer-based scaffolding is used can vary 
widely. Such variation is by learner population (e.g., grade level and other charac- 
teristics such as achievement level and socioeconomic status), subject matter (i.e., 
science, technology, engineering, and mathematics), and instructional model with 
which scaffolding is used (e.g., design-based learning and problem-based learning). 
I describe these variations, and note accompanying variations in effect size esti- 
mates. Notably, scaffolding had its strongest impact when students were (a) at the 
adult level, (b) engaged in project-based learning or problem solving and (c) from 
traditional learner populations. 


Keywords Case-based learning - Context of use : Design-based learning - Education 
level - Education population > Grade level - Instructional model - Inquiry-based 
learning - Modeling/visualization - Problem-based learning > Problem-centered 
instruction - Project-based learning - STEM discipline - Student demographics 


3.1 Rationale for this Chapter 


To begin this chapter, it is important to discuss the need for a consideration of 
the context of use of computer-based scaffolding. After all, in its original defini- 
tion, scaffolding was provided on a one-to-one basis to toddlers who engaged in 
unstructured problem-solving (Wood, Bruner, & Ross, 1976). All structure to the 
problem-solving activity was provided by the scaffolding process itself. This was 
practical when there was one teacher for each student, but lost its practicality as a 
single source of support when using scaffolding in formal schooling. After all, when 
a teacher can work on a one-to-one basis with one student for an unlimited time 
span, the teacher can continually assess what structure is needed, and provide it. 
This is hard to beat in terms of effectiveness. But in formal school settings, teach- 
ers very rarely have this opportunity. So, as researchers turned their attention to 
how instruction could be centered on problem-solving in formal education, it was 
important to think about additional ways to provide structure to student learning in 
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this context (Palincsar & Brown, 1984; Schmidt, Rotgans, & Yew, 2011). This was 
often accomplished by pairing scaffolding with formal problem-centered instruc- 
tional models (e.g., inquiry-based learning and problem-based learning; Crippen & 
Archambault, 2012; Hmelo-Silver, Duncan, & Chinn, 2007; Kolodner et al., 2003). 
Such formal, problem-centered instructional models needed to be paired with sup- 
port for students’ reasoning abilities, and instructional scaffolding (one-to-one and, 
later, computer-based and peer scaffolding) fit such a need nicely. 

A natural question is whether the specific problem-centered instructional model 
with which scaffolding is used influences scaffolding’s efficacy. This is an empiri- 
cal question. It is beyond the scope of this book to investigate variations in the 
efficacy of one-to-one scaffolding and peer scaffolding based on the specific prob- 
lem-centered instructional model with which it is used. But I do investigate how 
the efficacy of computer-based scaffolding varies based on the problem-centered 
instructional model with which it is used. 

Deploying scaffolding in formal education environments also entailed an expan- 
sion of the age groups with which scaffolding was used. Computer-based scaffold- 
ing is now used among learner populations at the elementary school, middle school, 
high school, university, graduate school, and adult levels. It is natural to question 
whether an instructional approach that was originally designed for toddlers, and 
then modified to allow it to be delivered via a computer-based tool, would be effica- 
cious among these new learner populations, and how the efficacy compares among 
the different learner groups. This is again an empirical question. 

Along with age/grade level, it is also important to consider the area of STEM 
in which scaffolding was used. Computer-based scaffolding is used in science, 
technology, engineering, and mathematics education. Is scaffolding more effective 
when used in the context of one of the STEM disciplines than the remaining STEM 
disciplines? This is an empirical question that I address in this chapter. 

Another important empirical question related to the expansion of the scaffolding 
metaphor to formal education is whether the efficacy of scaffolding varies depend- 
ing on the specific characteristics of the learners who use it. For example, does the 
influence of scaffolding vary according to prior achievement, socioeconomic status 
(SES), or other factors? Some research suggests that it does (Belland, 2010; Bel- 
land, Glazewski, & Richardson, 2011; Belland, Gu, Armbrust, & Cook, 2015; Cue- 
vas, Fiore, & Oser, 2002). Knowing the answer to this question would help scaffold- 
ing researchers know where further research is needed to improve outcomes among 
all students, an important goal to ensure that STEM is for all students (Lynch, 2001; 
Marra, Peterson, & Britsch, 2008; National Research Council, 2007). 

In the next sections, I first discuss research on computer-based scaffolding with 
an eye on grade level of the learner population, and then summarize the results of 
meta-analysis regarding differences in effect sizes of scaffolding according to grade 
level. Next, I discuss variations in the use of scaffolding according to STEM disci- 
pline, and differences in effect sizes on that basis. Subsequently, I discuss scaffold- 
ing literature in light of student demographics, and then note meta-analysis findings 
regarding differences in scaffolding’s effect according to student demographics. 
Next, I discuss how scaffolding is used in the context of different problem-centered 
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instructional approaches, and note any according variations in scaffolding’s effec- 
tiveness. 


3.2 Grade Level 


In a large expansion from the original grade level among which instructional scaf- 
folding was used in its original conceptualization—preschool (Wood et al., 1976)— 
scaffolding has come to be used at the primary, middle, secondary, college, gradu- 
ate, and adult levels (see Fig. 3.1). This likely makes sense in light of Adaptive 
Character of Thought-Rational (ACT-R) theory, which does not limit the scope of 
learners with whom it concerns itself. But this also makes sense in light of activity 
theory. While one traditionally may associate activity theory with learning among 
the pre-K-12 population, it is clear that the founders of activity theory never in- 
tended such a limitation in scope (Leont’ev, 1974; Luria, 1979). Rather, much of 
the core empirical research supporting activity theory involved adult populations 
(Luria, 1976). And the idea that one learns in interaction with others, in part by as- 
similating cultural knowledge, resonates with much other research on adult learning 
(Coryell, 2013). One may find the most clear such delimitation of a scaffolding- 
related learning theory in knowledge integration, which generally focuses on the 
learning of K-12 students (Linn, 2000). However, research from the knowledge 
integration tradition has been applied to older populations, and it is clear that there 
is a need for integrated mental models at all levels of education, and that many 
students at the college, graduate, and adult levels lack this (Ifenthaler & Seel, 2013; 
Johnson-Laird, 2001). 

At the same time, one would be remiss to think that adults and elementary school 
students, for example, would respond in exactly the same way to computer-based 
scaffolding. Computer-based scaffolding used among these different populations 
often varies to a great extent, but sometimes researchers use the same scaffold- 
ing for distinctly different student populations, such as graduate and middle school 


Preschoo! 


Fig. 3.1 The expansion of grade levels with which scaffolding is used from its original context of 
use (preschool) 
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students (Fretz et al., 2002; B. Zhang, Liu, & Krajcik, 2006) or college and middle 
school students (Kyza & Edelson, 2005; Land & Zembal-Saul, 2003). 

It is natural to question whether scaffolding has an effect of similar magnitude 
among learners at different grade levels. 


3.2.1 Results from the Meta-Analysis 


The scaffolding meta-analysis (Belland, Walker, Kim, & Lefler, In Press) 
included outcomes from the following levels: primary (71,, ,.4mc,= 28)» middle 
(Aovicomes= 198), high school (1,,,,4mes 753), College (1,,, comes 132), graduate 
school (7 aromes 11), and adult (11, mes 1) (See Table 3.1). Scaffolding had 
a statistically significantly greater effect among adult learners than among col- 
lege learners, high school students, middle level students, and primary students, 
p<0.01. Caution is warranted as the effect size estimate for adult learners is 
based on one outcome. Still, this is intriguing, in that one might have ventured to 
guess that the effect would be lowest among adults, given that scaffolding was 
originally developed for use among toddlers. At the same time, it is important to 
recall that in its original definition, instructional scaffolding referred to one-to- 
one interactions (Wood et al., 1976). 

Due to the higher sample size of effect sizes among middle level students than 
among graduate level learners, the 95% confidence interval for scaffolding’s effect 
among middle level learners (0.29-0.46) was tighter than it was for scaffold’s ef- 
fect among adult learners (0.20-1.52). Thus, the true effect size for adult learners 
may be lower than 0.86. From an activity theory perspective, scaffolding aims to 
help learners gain the cultural knowledge that helps to solve target problems effec- 
tively (Belland & Drake, 2013; Jonassen & Rohrer-Murphy, 1999). This is certainly 
something that graduate students and adults need to do. Still, the exact reason the 
effect size estimate is significantly greater among adult learners than among other 
age groups is unclear. 

It is important to recall that the fact that scaffolding had a statistically significant- 
ly greater effect among adult learners than among students from other age groups 


Table 3.1 Table of results of moderator analyses on the effect of education level on cognitive 
outcomes 


95 % Confidence interval 
Level n outcomes Effect size estimate Lower limit | Upper limit 
Adult 1 0.86 0.20 1.52 
Graduate 11 0.61 0.22 | 1.00 
College 132 0.49 0.42 0.57 
High school 53 | 0.48 0.34 0.62 
Middle school 108 | 0.37 0.29 0.46 
Elementary 28 0.55 0.40 0.67 
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does not mean that scaffolding’s effect was negative or inconsequential among the 
latter. Indeed, the effect size estimates of scaffolding used by elementary, middle, 
high school, college, and graduate level learners range from 0.37 to 0.61, which is 
above the threshold suggested for practical significance (Gall, Gall, & Borg, 2003), 
is substantially larger than the average effect of educational technology interven- 
tions for mathematics education (ES=0.16; Cheung & Slavin, 2013), and is signifi- 
cantly greater than zero. Furthermore, it is similar to the average effect of interven- 
tions designed to enhance critical thinking abilities among a wide range of learners 
(ES=0.341; Abrami et al., 2008), and higher than that of interventions designed to 
enhance critical thinking abilities among college students (ES=0.195; Niu, Behar- 
Horenstein, & Garvan, 2013). In short, scaffolding led to effect sizes that were 
significantly greater than zero, and practically significant, among individuals at the 
elementary, middle, secondary, college, graduate, and adult levels. For one inter- 
vention to be so robust to differences in student populations, and to so uniformly 
lead to positive effects, is rare in educational research. 

One may ask if the scaffolding interventions in the included research were simi- 
lar enough to all be called scaffolding. The lack of precision in the term scaffolding 
that had emerged throughout its expansion has been widely lamented (Pea, 2004; 
Puntambekar & Hübscher, 2005). The scaffolding definition that guided the under- 
lying meta-analysis was 

Support that assists students as they generate solutions to complex and ill-structured tasks, 

problems, or goals, and increases and helps students integrate higher-order competencies, 


including problem solving skills, deep understanding of content (knowledge integration), 
or argumentation. 


This definition was applied strictly. For example, articles in which the intervention 
was given to students before they engaged in the problem were excluded, as were 
articles in which students were not addressing authentic, ill-structured problems or 
tasks. But there is clearly room for variation in the scaffolding interventions pro- 
vided that they met the scaffolding definition. 


3.3 STEM Discipline 


Though STEM content was not central to the original instructional scaffolding defi- 
nition (See Fig. 3.2), scaffolding has grown to be a central instructional strategy 
used in conjunction with problem-centered instruction in STEM education (Crippen 
& Archambault, 2012). The problem-centered instructional models used in each 
of these disciplines often vary. For example, modeling/Vvisualization tends to be 
used most often in engineering and mathematics education (Lesh & Harel, 2003; 
Vreman-de Olde & de Jong, 2006). Design-based learning tends to be used most 
often in engineering education or in science education integrated with engineering 
content (Gómez Puente, Eijck, & Jochems, 2013; Kolodner et al., 2003; Mehalik, 
Doppelt, & Schuun, 2008). Problem-based learning is often used in science and en- 
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Not applicable 


Mathematics 


Fig. 3.2 The expansion of disciplines of instruction in which scaffolding is used, going from non- 
STEM to science, technology, engineering, and mathematics 


gineering education (Belland, 2010; Carr, Bennett, & Strobel, 2012; Hmelo-Silver, 
2004). Furthermore, the types of skills being supported and content being devel- 
oped varies among the disciplines. For example, design-based learning is prominent 
in engineering education because engineering places such a strong emphasis on the 
design of solutions to address problems. 


3.3.1 Results from the Meta-Analysis 


Outcomes from science (N,,, comes 298), technology (Njjeomes= 21), engineer- 
ing (M urcomes 730), and mathematics education (7 uicomes 744) were included (See 
Table 3.2; Belland et al., In Press). Results indicate that there was no differ- 
ence in scaffolding’s effect on the basis of discipline. This suggests that scaffolding 
is a robust intervention that is highly effective when used solving problems in a 
variety of subject matters. 


3.4 Student Demographics 


The original scaffolding description was developed among traditional, middle class 
students (See Fig. 3.3; Wood et al., 1976). In the scaffolding literature, one often 
sees variations in cognitive outcomes from scaffolding based on student factors 
such as achievement level, SES, and other factors associated with underrepresenta- 


Table 3.2 Table of results of moderator analyses on the effect of STEM discipline on cognitive 
outcomes 


95% Confidence interval 
Level n outcomes Effect size estimate Lower limit | Upper limit 
Science 208 0.42 0.36 0.48 
Technology 51 0.51 0.36 0.67 
Engineering 30 0.58 0.42 0.73 
Mathematics 44 0.54 0.42 0.65 
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Under-performing 
earning disabilities 


Fig. 3.3 The expansion of student populations with which instructional scaffolding is used, from 
traditional to traditional, low-income, underrepresented, high-performing, underperforming, and 
student with learning disabilities 


tion in STEM (Azevedo, Winters, & Moos, 2004; Belland, 2010; Belland et al., 
2011; Belland, Gu, Armbrust, et al., 2015; Belland, Gu, Kim, Turner, & Weiss, 
2015; Cuevas et al., 2002; Simons & Klein, 2006). It is important to investigate the 
extent to which scaffolding’s influence varies according to these variables to guide 
future scaffolding research and development, so as to help ensure that STEM is for 
all (Lynch, 2001; National Research Council, 2007). 


3.4.1 Results from the Meta-Analysis 


Outcomes from the following learner populations were included: traditional (,,, 
comes 279), low income (1, joes 11), underrepresented (7 utcomes 17), high- 
performing (71, oes 8)» and underperforming (7 mes 18) (See Table 3.3; 
Belland et al., In Press). Students from traditional learner populations had 
a Statistically significantly higher average effect size (g=0.48) than underper- 
forming students (g=0.28), p<0.05. This is concerning, as it is very important 
to maximize success opportunities in STEM for students from underrepresent- 
ed groups (Ceci, Williams, & Barnett, 2009; National Research Council, 2011; 
Thoman, Smith, Brown, Chase, & Lee, 2013). Further research is needed to ex- 
amine how to design and deploy computer-based scaffolding so as to increase its 
efficacy among underrepresented groups. There may also be a need to develop 
versions of scaffolds that draw on strategies known to be effective among the 
underrepresented groups (Cuevas et al., 2002; Lynch, 2001; Marra et al., 2008). 
It is clear from the literature that this is possible, as some studies have shown 
that specific scaffolds are more effective among lower-achieving and lower-SES 
middle school students than among higher-achieving and average-to-higher-SES 
students (Belland et al., 2011; Belland, Gu, Armbrust, et al., 2015; Belland, Gu, 
Kim, et al., 2015). 
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Table 3.3 Table of results of moderator analyses on the effect of student demographics on cogni- 
tive outcomes 


95 % Confidence interval 
Level n outcomes Effect size estimate | Lower limit | Upper limit 
High performing 8 0.36 0.07 0.66 
Low income 11 0.51 0.32 0.70 
Traditional 279 0.48 0.42 0.53 
Underperforming 18 0.28 0.12 0.45 
Underrepresented 17 0.41 0.17 0.66 


3.5 Instructional Model with Which Scaffolding is Used 


In the original formulation of the scaffolding definition, no thought was given to 
the instructional model with which scaffolding was used, as it was centered on one- 
to-one tutoring of toddlers learning to build pyramids with wooden blocks (Wood 
et al., 1976). But as scaffolding was applied to formal education, one needed to 
consider the problem-centered instructional model with which scaffolding would 
be used (See Fig. 3.4). Scaffolding can be used in the context of such instructional 
strategies as problem-based learning, case-based learning, design-based learning, 
inquiry-based learning, project-based learning, and other instructional approaches 
that engage students in problem-solving. It is natural to question whether scaffold- 
ing’s effectiveness varies according to the problem-centered instructional model 
with which it is used. There is reason to believe that it may, because different prob- 
lem-centered models have different levels of structure and support for students built 
into their approach. The underlying support of the instructional model could interact 
with the support of scaffolding in a positive or negative way. 


Not applicable 
Problem-based learning 


ase-Dased learning 


Fig. 3.4 The expansion of formal, problem-centered models with which to use instructional scaf- 
folding, from none to problem-based learning, case-based learning, design-based learning, project- 
based learning, inquiry-based learning, and problem-solving 
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3.5.1 Problem-Based Learning 


Problem-based learning is an instructional approach in which students are presented 
with an authentic, ill-structured problem, and need to determine what they already 
know about the problem and what they need to know (learning issues; Barrows & 
Tamblyn, 1980; Hmelo-Silver, 2004). Typically, teachers present a driving question 
such as, “How does water quality affect the flora and fauna of X valley?” to which 
students can refer throughout the unit, and which reminds them of the fundamen- 
tal reason they are addressing the problem (Ertmer & Simons, 2006). After being 
presented with the problem, defining it, and generating learning issues, students 
proceed to address their learning issues, and then develop a potential solution and 
back it up with evidence (Belland, Glazewski, & Richardson, 2008; Hmelo-Silver, 
2004). They then need to defend their solution (Belland et al., 2008). 

Originally developed in the medical school context, problem-based learning is 
still used extensively there (Barrows & Tamblyn, 1980; Gijbels, Dochy, Van den 
Bossche, & Segers, 2005; Kalaian, Mullan, & Kasim, 1999). In this setting, simu- 
lated patients often present with an unidentified illness, and students need to re- 
search what might cause such symptoms, triangulate such with test results, and 
propose a diagnosis and treatment (Barrows, 1985; Hmelo et al., 2001). Problem- 
based learning is also used in other university contexts such as business (Arts, Gi- 
jselaers, & Segers, 2002; Giesbers, Rienties, Tempelaar, & Gijselaers, 2013) and 
teacher education (Hmelo-Silver, Derry, Bitterman, & Hatrak, 2009; McCormick 
Peterman, 2012), as well as in various K-12 contexts. For example, in high school 
social studies, students addressed historical problems in the civil rights era (Saye 
& Brush, 2002). Middle school science students addressed a problem related to 
genetic testing and its relationship with such issues as medical insurance and public 
health (Belland, 2010; Belland et al., 2011). Furthermore, middle school students 
addressed the evolution of water quality in a local river, and what should be done 
about it (Belland, Gu, Armbrust, et al., 2015; Belland, Gu, Kim, et al., 2015). Col- 
lege statistics students investigated the extent to which claims related to a presented 
problem were supported by statistics (Karpiak, 2011). Preservice teachers investi- 
gated typical classroom problems, and developed solutions using educational psy- 
chology content (Hmelo-Silver et al., 2009). In these settings, problems are often 
presented through text-based or video-based synopses of the problem (Hmelo-Sil- 
ver, 2004; Hung, Jonassen, & Liu, 2007). 

In problem-based learning, students most often work in small groups. Though 
the sizes of the groups sometimes vary, 3—4 students is often posited as an ideal 
size in terms of promoting maximum student discussion (Arts et al., 2002; Lohman 
& Finkelstein, 2000). Different members of groups often perform different roles 
based on their individual strengths, and this can serve to extend each student’s capa- 
bilities (Belland, Glazewski, & Ertmer, 2009; Helle, Tynjälä, & Olkinuora, 2006). 
However, there is some evidence that problem-based learning can be effective even 
when students work individually (Pease & Kuhn, 2011). 
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Problem-based learning both requires the use of strong self-regulated learning 
skills on the part of students, and can often lead to the enhancement of self-regu- 
lated learning skills (Evensen, Salisbury-Glennon, & Glenn, 2001; Loyens, Magda, 
& Rikers, 2008). But it also requires that students identify learning issues related 
to what they need to know to solve the problem (Hmelo-Silver, 2004; Loyens et 
al., 2008). Many K-12 students lack sophisticated self-regulated and self-directed 
learning skills and so need to be supported in these areas through scaffolding (Aze- 
vedo, 2005; Loyens et al., 2008). Similar struggles with self-directed learning can 
be seen among college students (Lekalakala-Mokgele, 2010) and medical students 
(Lohman & Finkelstein, 2000). Perhaps due to problem-based learning’s focus on 
self-regulated learning, being exposed to problem-based learning and accompany- 
ing one-to-one scaffolding led seventh grade science students to develop signifi- 
cantly and substantially more enhanced epistemic beliefs (Belland, Gu, Kim, et al., 
2015). 

Problem-based learning leads to strong learning outcomes. For example, meta- 
analyses indicate that problem-based learning has a strong effect on principles-level 
(Gijbels et al., 2005) and application-level (Walker & Leary, 2009) outcomes and 
long-term retention (Strobel & van Barneveld, 2009). At the principles level, stu- 
dents performed on average 0.795 standard deviations better than their control 
counterparts (Gijbels et al., 2005). The advantage at the application level was 
0.334 standard deviations (Walker & Leary, 2009). Given that the Strobel and van 
Barneveld’s (2009) paper was a meta-synthesis, a quantitative estimate of the effect 
size difference is not available. Problem-based learning has often been found to 
lead to weaker immediate recall than lecture (Albanese & Mitchell, 1993; Dochy, 
Segers, Van den Bossche, & Gijbels, 2003; Kalaian et al., 1999) but better long-term 
retention and deep content learning than lecture (Belland, French, & Ertmer, 2009; 
Pourshanazari, Roohbakhsh, Khazaei, & Tajadini, 2013; Strobel & van Barneveld, 
2009). 


3.5.2 Case-Based Learning 


Case-based learning is often used in the law school and business school contexts. 
But it also has been used in such STEM disciplines as medicine (Thistlethwaite et 
al., 2012) and physics (J. Zhang, Chen, Sun, & Reid, 2004). Lectures on the neces- 
sary content to understand the case often precede the presentation of cases. The 
premise is that by providing cases, instruction can help students build up a reper- 
toire of cases upon which learners can draw when encountering similar problems 
in the future (Jonassen & Hernandez-Serrano, 2002; Kolodner, 1993). Cases can 
also provide concrete contexts in which the new content can be applied. Cases can 
represent a business transition or response to a problem or a particularly cogent 
legal case/argument/decision. Cases are often presented in a group discussion con- 
text, but can also take the form of a video summary or an online case presentation 
(Thistlethwaite et al., 2012). In it, learning content to be covered in the case (often 
via listening to a lecture) happens before students engage with the case. Typically, 
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there is not much content to be learned after being presented with the case, but 
rather students need to reason based on what they have already learned (Srinivasan, 
Wilkes, Stevenson, Nguyen, & Slavin, 2007). Furthermore, faculty give students 
more active guidance than they would in a problem-based learning approach (Srini- 
vasan et al., 2007). In this way, on the continuum of problem-centered approaches 
to instruction, case-based learning is closer to the more guided side than to the 
less guided side (Srinivasan et al., 2007). While cases represent authentic prob- 
lems, they are typically more context bound than problem-based learning problems 
(Jonassen, 2000; Savery, 2006). In addition, cases are used to assess learning and 
promote application, rather than to drive learning (Savery, 2006). 

The relative sophistication of students’ epistemic beliefs influence their ability 
to perform well in a case-based learning environment, with students with sophis- 
ticated epistemic beliefs performing better and benefitting more from scaffolding 
than students with unsophisticated epistemic beliefs (Demetriadis, Papadopoulos, 
Stamelos, & Fischer, 2008; Peng & Fitzgerald, 2006). Some evidence indicates that 
case-based instruction can also help students develop more sophisticated epistemo- 
logical beliefs (Cam & Geban, 2011). 

Systematic reviews of the literature on the use of case-based learning in medical 
education indicates that students and instructors like the method very much, but 
how its impact on learning compares with that of other methods is not conclusive 
(Srinivasan et al., 2007; Thistlethwaite et al., 2012). 


3.5.3 Design-Based Learning 


In design-based learning, students are presented with an authentic, ill-structured 
problem, but rather than develop a conceptual solution, they need to design/en- 
gineer a product that addresses the problem (e.g., a levee to prevent erosion on a 
barrier island (Kolodner et al., 2003), an alarm to address a problem that students 
identified (Silk, Schunn, & Cary, 2009)). Such problems are usually drawn from 
students’ immediate communities, and students often have an opportunity to iden- 
tify a specific subproblem on which they want to work (Doppelt & Schunn, 2008; 
Duran, Höft, Lawson, Medjahed, & Orady, 2014). The central problem in this ap- 
proach is often termed a design challenge (Brophy, Klein, Portsmore, & Rogers, 
2008). To address design challenges, it is important to consider the goals as envi- 
sioned by various project stakeholders, as well as constraints governing the design 
of a solution (Brophy et al., 2008). For example, design challenges can include 
preventing erosion on barrier islands and designing a model car that can go up and 
down hills on a track (Kolodner et al., 2003). In another approach to design-based 
learning, students generate a design challenge related to security alarms, taking into 
account where they personally need an alarm system (e.g., to remind someone to 
take medicine or to alert that something has been stolen (Doppelt, Mehalik, Schunn, 
Silk, & Krysinski, 2008)). In the process of designing the product, students need to 
address learning issues (Chandrasekaran, Stojcevski, Littlefair, & Joordens, 2013; 
Kolodner et al., 2003; Silk et al., 2009). 
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To my knowledge, the effect of design-based learning has not been investigated 
through meta-analysis. But individual empirical studies indicate that design-based 
learning leads to many beneficial outcomes. For example, middle school students 
engaged in design-based learning have been found to learn science content and 
problem-solving skills more effectively than typical instruction controls (Kolodner 
et al., 2003) and gain substantially in science inquiry skills from pre to post (Silk et 
al., 2009). Furthermore, design-based learning led to significant increases in content 
knowledge and core STEM process skills among high school students (Duran et al., 
2014). 


3.5.4 Inquiry-Based Learning 


Inquiry-based learning is characterized by overt foci on students (a) posing their 
own questions early in the process (Edelson, Gordin, & Pea, 1999) and (b) design- 
ing and carrying out an experimental technique to address the generated questions 
(Abd-El-Khalick et al., 2004; Gibson & Chase, 2002). In this way, inquiry-based 
learning differs markedly from the “rhetoric of conclusions” approach to science 
labs used in many science classes, in which students are presented a question for 
which scientists know the answer quite well and given experimental procedures to 
follow to address that question (Chinn & Malhotra, 2002; Duschl, 2008). Rather, in 
inquiry-based learning, students need to identify variables, state hypotheses, design 
and carry out tests of those hypotheses, and interpret and explain the results (Edel- 
son et al., 1999; Jong, 2006; Minner, Levy, & Century, 2010). There is substantial 
guidance from teachers and technology along the way, for example, for identifying 
pertinent variables and formulating testable hypotheses (Jong, 2006; Keys & Bryan, 
2001). For example, high school students were invited to interact with a climate 
visualization, in which they could identify questions they wanted to address and 
manipulate variables to see how that affected dependent variables (Edelson et al., 
1999). In another example, high school students interacted with an astronomy vi- 
sualization with which they could address ten questions by manipulating variables 
(Taasoobshirazi, Zuiker, Anderson, & Hickey, 2006). 

According to a recent meta-analysis of the literature on inquiry-based instruction 
in science, the model led to an average effect of 0.5, a medium-large effect (Furtak, 
Seidel, Iverson, & Briggs, 2012). Notably, the effect sizes in the review were twice 
as big when teacher support was highest (Furtak et al., 2012). According to another 
review, it may not be inquiry-based learning per se that leads to strong learning out- 
comes, but the extent to which students need to analyze authentic data and generate 
conclusions (Minner et al., 2010). 

Examined individually, empirical studies indicate that inquiry-based learning 
can help students develop inquiry skills, as well as deep content learning (Crippen 
& Archambault, 2012; Edelson et al., 1999; Marx et al., 2004). Inquiry-based learn- 
ing can be a good strategy to help students in underperforming districts perform 
at a higher level, when deployed as part of systematic reform (Marx et al., 2004). 
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Furthermore, inquiry-based learning can promote enhanced and sustained interest 
in science (Gibson & Chase, 2002). An extensive review of the inquiry-based learn- 
ing literature indicated that it may be the extent to which students need to actively 
think, rather than the model of inquiry-based learning in and of itself, that leads to 
enhanced content learning (Minner et al., 2010). 


3.5.5 Project-Based Learning 


In project-based learning, students address a problem, but the central focus is on 
the product that students need to create (Helle et al., 2006; Krajcik et al., 1998). 
Some examples of products are a video, a PowerPoint presentation, or a report. 
In developing project-based learning curricula, designers list academic standards, 
specify what students should be able to do according to the standard, and devise a 
performance (project) that would provide evidence of mastery of the skill (Barron et 
al., 1998; Krajcik et al., 1998). For example, a project-based learning unit in middle 
school invited students to design blueprints for a playhouse, given a set of donated 
materials (Barron et al., 1998). Students then work toward the project, which is typi- 
cally contextualized in some sort of problem that students have the potential to find 
engaging (Krajcik, McNeill, & Reiser, 2008). As with problem-based learning, a 
driving question typically guides student learning in project-based learning (Barron 
et al., 1998). A primary purpose of a driving question in this case is to keep student 
focus on the content being learned and the issues being addressed, rather than on the 
project per se (Barron et al., 1998). While the parameters of the project deliverable 
are given to students at the beginning of the unit, students typically have a substan- 
tial amount of freedom in determining the exact features of the deliverable, as well 
as the route to get there (Helle et al., 2006). However, project-based learning is 
typically more structured than problem-based learning in that its deliverable is more 
well-specified (Savery, 2006). At the end of project-based learning, students typi- 
cally produce the target product, and then engage in some sort of reflection, which 
can include the creation of a portfolio (Turns, Cuddihy, & Guan, 2010). 

Research on project-based learning is often focused more on curricular design 
than on student learning (Helle et al., 2006). However, an examination of the proj- 
ect-based learning literature can lead one to some observations. First, project-based 
learning can lead to strong gains in design skills on the part of elementary school 
students (Barron et al., 1998) and college students (Dym, Agogino, Eris, Frey, & 
Leifer, 2005). It is also an instructional approach that can be very motivating (Blu- 
menfeld et al., 1991; Helle, Tynjälä, Olkinuora, & Lonka, 2007). However, project- 
based learning does not necessarily promote motivation in and of itself; rather, de- 
signers should take care to design projects so as to enhance and sustain motivation 
(Blumenfeld et al., 1991) and to design scaffolding that supports motivation (Bel- 
land, Kim, & Hannafin, 2013). 
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3.5.6 Other Instructional Approaches 


Scaffolding can be incorporated into other instructional approaches that incorpo- 
rate authentic problem-solving but do not fit the above labels. This approach will 
be called problem-solving instruction for the purposes of this book. For example, 
much work in intelligent tutoring systems does not fit within any of the above in- 
structional models, as it is grounded in the ACT-R theory of learning (Anderson, 
Matessa, & Lebiere, 1997). However, much of it does involve authentic problem- 
solving. As noted in Chap. 2 (this volume), intelligent tutoring systems focus on 
delivering knowledge chunks to students that they can then apply to problems that 
are provided in sequence. Scaffolding within Intelligent Tutoring Systems focuses 
on helping students apply the content to the problems, and in the process generate 
production rules. Production rules are defined as rules governing the application 
of declarative content to problems that can be applied without conscious control to 
similar problems in the future (Koedinger & Corbett, 2006; Self, 1998). 

Many intelligent tutoring systems are used in mathematics. For example, the Ge- 
ometry Cognitive Tutor presents a series of geometry problems along with diagrams 
(Aleven & Koedinger, 2002). Students need to calculate things like angles and type 
explanations of how they got their answer. They are given feedback on the basis of 
their answer and explanations (for the most common mistakes, detailed feedback is 
provided). Students can also request hints. 


3.5.7 Results from the Meta-Analysis 


The meta-analysis included outcomes in which scaffolding was used in the con- 
text of problem-based learning (71, mes 38)» Case-based learning (71, .4mes= 15) 
modeling/Vvisualization (11, james 42)» Project-based learning (71, ,4mes 75), design- 
based learning (11, joes 4)» inquiry-based learning (71, jomes 709), and problem- 
solving (11, comes 160) (See Table 3.4; Belland et al., In Press). Results indi- 
cated that scaffolding utilized in the context of project-based learning had a higher 
average effect size (g= 1.33) than scaffolding used in the context of problem-based 


Table 3.4 Table of results of moderator analyses on the effect of problem-centered model on 
cognitive outcomes 


95 % Confidence interval 
Lower limit 


Level n outcomes Effect size estimate 


Upper limit 
Case-based learning 


Design-based learning 4 | 0.30 —0.12 0.82 
Inquiry-based learning 69 | 0.42 0.33 0.52 
Modeling/visualization 42 [0.51 0.34 0.68 
Problem-based learning | 38 | 0.27 0.11 0.43 
Problem-solving 160 [0.53 0.45 0.58 


Project-based learning 5 1.33 1.03 1.63 
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learning (g=0.27), problem-solving (g=0.53), modeling/visualization (g=0.51), 
design-based learning (g=0.30), inquiry-based learning (g=0.42), and case-based 
learning (g=0.28), p<0.01. Furthermore, scaffolding used in the context of prob- 
lem-solving had a higher effect size estimate (g=0.53) than scaffolding used in 
the context of problem-based learning (g=0.27), p<0.01. Still, this difference is 
borderline, as the 95 % confidence intervals overlap. 

Of note, most studies that used the “problem-solving” instructional approach 
involved intelligent tutoring systems informed by the ACT-R theory (Anderson 
et al., 1997; Corbett & Anderson, 2001; Koedinger & Corbett, 2006). A previous 
meta-analysis found that step-based intelligent tutoring systems led to an average 
effect size of 0.76 (VanLehn, 2011), which is considerably larger than the average 
effect size of computer-based scaffolding from the current, underlying, scaffolding 
meta-analysis (g=0.46). This does not mean that intelligent tutoring systems are 
superior to other scaffolding types, as they target a different form of learning than 
other scaffolding types and hold different assumptions about learning and ways to 
help people learn most effectively. Notably, intelligent tutoring systems are the most 
highly structured instructional programs that involve scaffolding, in that they care- 
fully script all student actions, and what happens when students do particular ac- 
tions (Koedinger & Aleven, 2007; Koedinger & Corbett, 2006). The only exception 
to this is typically the inclusion of a hint button, which students can choose to press 
or not. It may be that the structure of intelligent tutoring systems in conjunction with 
scaffolding helps produce effects of a larger magnitude. It is important to note that 
this inclusion of a very tight structure means that intelligent tutoring systems tend 
to minimize opportunities for self-directed learning. For further discussion of the 
theoretical bases of scaffolding, please see Chap. 2 (this volume). 

It is notable that the effect size estimate for computer-based scaffolding used 
in the context of inquiry-based learning (g=0.42) is below that of inquiry-based 
learning found in a recent meta-analysis (ES=0.50) by Furtak et al. (2012). How- 
ever, it is reasonably close. One may imagine that not all studies covered in the 
latter meta-analysis included computer-based scaffolds. Further research is needed 
to disentangle the effect of computer-based scaffolding and that of inquiry-based 
learning in this context. 

That the effect size of computer-based scaffolding was lowest when paired with 
problem-based learning may be explained by the open-ended nature of problem- 
based learning. Problem-based learning requires self-directed learning perhaps to 
the greatest extent among the covered problem-centered instructional approaches 
(Lohman & Finkelstein, 2000; Loyens et al., 2008; Savery, 2006). Problem-based 
learning students are responsible not only for defining the problem, but also de- 
termining what they need to know to come up with a solution, finding the infor- 
mation, and synthesizing the information to determine a solution (Belland et al., 
2008; Hmelo-Silver, 2004; Loyens et al., 2008). In short, students have less struc- 
ture from the inherent nature of problem-based learning than they would have from 
inquiry-, case-, project-, or design-based learning. Thus, they need to be relatively 
autonomous. This can be particularly challenging for K-12 students who have little 
experience with autonomy in school (Jang, Reeve, & Deci, 2010; Rogat, Witham, 
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& Chinn, 2014; Stefanou, Perencevich, DiCintio, & Turner, 2004). Furthermore, if 
teachers do not provide appropriate autonomy support, defined as the provision of 
meaningful choice in academic tasks and explanation when choice is not possible, 
students may not strive to achieve mastery, but rather, strive to perform better than 
other students (Ciani, Middleton, Summers, & Sheldon, 2010; Deci & Ryan, 2000). 
The nature of scaffolding used in the context of problem-based learning is thus 
uniquely targeted toward the need for students to be self-determined (Belland et al., 
2008; Hmelo-Silver et al., 2007). 

What problem-based learning students can propose as a solution is typically 
more open-ended than in case-based learning, project-based learning, design-based 
learning, modeling/visualization, and inquiry-based learning (Hmelo-Silver, 2004; 
Savery, 2006). That is, possible deliverables include conceptual solutions to the 
problem, persuasive presentations, artifacts, or some combination of products. In 
this way, problem-based learning may be seen as more loosely structured than other 
problem-centered instructional models (Hung, 2011; Savery, 2006). 
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Chapter 4 
Intended Learning Outcomes and Assessment 
of Computer-Based Scaffolding 


Abstract In this chapter, I describe the intended learning outcomes of scaffold- 
ing—content knowledge and higher-order thinking abilities—and link these to the 
goals advanced by the Next Generation Science Standards and related documents 
from recent curricular revisions in STEM education. Furthermore, I address different 
ways in which scaffolding’s effect can be measured (assessment level), and explore 
whether there are differences in the magnitude of scaffolding’s effect according to 
assessment level. Meta-analysis results show that there is no difference in effect 
size magnitude on the basis of intended learning outcome (i.e., content knowledge 
or higher-order thinking abilities). Scaffolding’s effect was greater when measured 
at the principles level than when measured at the concept level. But scaffolding’s 
effect was statistically greater than 0 and substantial for all three assessment levels 
(1.e., concept, principles, and application). These results are then discussed. 


Keywords Application-level assessment - Argumentation - Assessment levels 
- Common Core : Concept-level assessment - Epistemology > Intended learning 
outcomes - Next Generation Science Standards - Principles-level assessment + 
STEM education 


4.1 Rationale for this Chapter 


In science, technology, engineering, and mathematics (STEM) education, 
computer-based scaffolding has been deployed to help enhance students’ higher- 
order thinking skills (Belland, 2010; Cho & Jonassen, 2002; Eck & Dempsey, 2002; 
M. Kim & Hannafin, 2011) and deep content learning (Chang & Linn, 2013; Davis, 
2003; Hwang, Shi, & Chu, 2011). These diverse learning outcomes may be seen by 
some as evidence of two categorically different interventions that cannot be con- 
sidered alongside each other. But these dual emphases of scaffolding can be seen 
as congruent with the emphases on learning the process of STEM, as well as learn- 
ing cross-cutting concepts and disciplinary core ideas in the Next Generation Sci- 
ence Standards (NGSS; Achieve, 2013; National Science Board, 2010). Needless 
to say, scaffolding’s emphases did not emerge in direct response to the writing of 
the NGSS, as such emphases were formed well before the NGSS existed. Rather, 
scaffolding’s intended learning outcomes arose within and alongside the currents of 
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the transformation of education from a didactic process of information transfer to 
one of construction of knowledge. 

In this chapter, to provide context and to help the reader understand the seeming 
dichotomy of learning goals of scaffolding, I first situate scaffolding relative to the 
calls for the enhancement of content knowledge and higher-order thinking skills in 
the NGSS (Achieve, 2013; Krajcik, Codere, Dahsah, Bayer, & Mun, 2014; National 
Science Board, 2010) and the Common Core State Standards (McLaughlin & Over- 
turf, 2012; National Governors Association Center for Best Practices & Council of 
Chief State School Officers, 2010). 

Second, I expand on the intended learning outcomes of scaffolding. Variation 
in intended learning outcomes of scaffolding largely aligns with differences in the 
theoretical underpinnings of scaffolding, which were discussed in Chap. 2: “In- 
structional Scaffolding: Foundations and Evolving Definition.” I also explore if the 
effectiveness of scaffolding varies according to intended learning outcome, as in- 
formed by the meta-analysis results. 

Just as it is important to consider intended learning outcomes, it is also important to 
consider how learning is assessed (Belland, 2012; Belland, French, & Ertmer, 2009; 
Furtak & Ruiz-Primo, 2008; Messick, 1989). Indeed, one is often advised to consider 
assessment before even designing objectives and instructional materials/strategies 
(Gagné, 1965; Wiggins & McTighe, 2005). By considering how scaffolding’s influ- 
ence on cognitive outcomes varies according to how it is assessed—at the concept, 
principles, or application level (Sugrue, 1995)—one can see if scaffolding as a whole 
delivers stronger impacts on content learning or various types of higher-order think- 
ing skills. It is important to consider this alongside the intended learning outcome, 
as (a) just because an intervention is designed to increase content learning or higher- 
order skills does not necessarily mean that it does, and (b) just because scholars claim 
that scaffolding is intended to help students enhance their skill in a particular area 
does not always mean that the learning is being assessed at that level. 

In this chapter, I discuss these ideas, and present meta-analysis results comparing 
scaffolding’s impact according to intended learning outcome and assessment levels. 


4.2 Targeted Learning Outcomes of Scaffolding 


Scaffolding has been designed to promote higher-order skills such as ill-structured 
problem-solving ability (Ge & Land, 2004; Liu & Bera, 2005) and argumentation 
ability (Belland, Gu, Armbrust, & Cook, 2015; McNeill & Krajcik, 2009), and en- 
hanced/deep content knowledge (Davis & Linn, 2000; Koedinger & Corbett, 2006). 
It is important to note that in the intelligent tutoring systems literature, authors posit 
a focus on enhancing procedural knowledge (production rules) by which individu- 
als can apply declarative knowledge. Some may argue that this is a form of prob- 
lem-solving skill. But I argue that it is a form of content learning, as each production 
rule is concerned with how to apply one highly specific domain knowledge element 
(Anderson et al., 2004). 
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Fig. 4.1 The relationship between the intended learning outcomes of instructional scaffolding and 
of the Next Generation Science Standards (Achieve, 2013) 
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The interrelationship between the intended learning outcomes of scaffolding and of 
the NGSS are illustrated in Fig. 4.1 and expanded upon in the sections that follow. 


4.2.1 Higher-Order Thinking Skills 


4.2.1.1 Ill-Structured Problem-Solving Ability 


Scaffolding to promote problem-solving ability is closest to the original instruc- 
tional scaffolding definition (Wood, Bruner, & Ross, 1976). Problem-solving abil- 
ity in this case refers to the ability to solve ill-structured problems—problems with 
many possible valid solutions and many valid solution paths (Jonassen, 2000, 2011). 

To be successful solving ill-structured problems, learners need to qualitatively 
model such problems so that they can determine what entities interact in the prob- 
lem, how they interact, and what such interaction means to each entity (Chi, Fel- 
tovich, & Glaser, 1981; Jonassen, 2003; Klahr & Simon, 1999; Lesh & Harel, 2003; 
Nersessian, 2008). But then they need to characterize the disparity between the 
goal state and the current state and determine an appropriate way to bridge the gap 
(Jonassen, 2000). However, this process is different from the means-ends analysis 
that describes how people often solve well-structured problems. Rather, solving ill- 
structured problems is an iterative process of defining the problem and identifying 
and weighing potential goal states and different methods of arriving at those goal 
states (Chi et al., 1981; Giere, 1990; Jonassen, 2000, 2003; Nersessian, 2008). By 
definition, ill-structured problems often have many solutions that are equally valid 
(Jonassen, 2011). In this way, the suitability of solutions to ill-structured problems 
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needs to be judged on the basis of evidential support (Belland, Glazewski, & Rich- 
ardson, 2008; Ford, 2012; Jonassen & Kim, 2010). Students thus need to have the 
opportunity to build and evaluate evidence-based arguments to be able to engage 
in ill-structured problem-solving, and to prepare for the modern workforce (Ford, 
2012; Gu & Belland, 2015; Jonassen, 2011; Osborne, 2010; Perelman & Olbrechts- 
Tyteca, 1958). 

The ability to solve ill-structured problems is qualitatively different from solving 
well-structured problems such as the story problems found in many mathematics 
textbooks (Jonassen, 2000; Lesh & Harel, 2003; Nersessian, 2008). One can solve 
well-structured problems with only the information given in the problem descrip- 
tion, whereas solving ill-structured problems requires the acquisition, evaluation, 
and use of much data beyond that given in the problem description. Well-structured 
problems have only one correct answer, and often only one solution path, whereas 
ill-structured problems have multiple potentially correct solutions, and many ways 
of arriving at them. Given these differences, the strategies by which one addresses 
well-structured problems and ill-structured problems differ (Jonassen, 2000). As 
such, one cannot promote the enhancement of ill-structured problem-solving ability 
by engaging students in well-structured problem-solving; rather, one should engage 
students in ill-structured problem-solving along with instructional support such as 
scaffolding (Abd-El-Khalick et al., 2004; Jonassen, 2011). 


4.2.1.2 Argumentation Ability 


Argumentation ability refers to the ability to back claims with evidence by way of 
premises, and evaluate and respond to the extent to which claims presented by oth- 
ers are well supported by evidence (D. Kuhn, 1991; Perelman & Olbrechts-Tyteca, 
1958; van Eemeren, Grootendorst, & Snoeck Henkemans, 2002). Two prominent 
models of argumentation are those of persuasive argumentation and dialectical 
argumentation. According to the former, there is no such thing as a universally 
valid argument; rather, arguments are successful to the extent to which the audience 
agrees with its central claim. As such, the goal of argumentation is to lead the audi- 
ence to adhere to the validity of one’s claim (Perelman & Olbrechts-Tyteca, 1958; 
Walton, 1989). In persuasive argumentation, novice arguers often focus on strength- 
ening one’s own position (D. Kuhn, 1991; Vellom & Anderson, 1999). 

Dialectical argumentation starts off with individuals creating evidence-based 
arguments, but from there it diverges. Specifically, rather than simply supporting 
one’s own claims, in dialectical argumentation, one also engages with claims of 
others (Asterhan & Schwarz, 2009; Jonassen & Kim, 2010; Keefer, Zeitz, & Resn- 
ick, 2000). This can include attempting to weaken the position of others (Asterhan 
& Schwarz, 2009; D. Kuhn, 1991) or negotiating with opposing parties in pursuit 
of an ultimate truth (Jonassen & Kim, 2010; Keefer et al., 2000; van Eemeren & 
Houtlosser, 2001). In the latter case, the opposing parties make concessions in their 
arguments in the service of improving their claims and ultimately moving toward an 
ultimate truth that is not directly knowable, but which can be approached through 
negotiation of arguments. 
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Argumentation can be considered a subset of problem-solving ability (Jonassen 
& Kim, 2010; D. Kuhn, 1991), and is the process by which scientific knowledge 
advances (Ford, 2012; Osbome, 2010). As discussed earlier, argumentation is core 
to how the quality of solutions to ill-structured problems is judged. Having arrived 
at initial solutions to such problems, argumentation is also how such solutions are 
iteratively improved, as well as the evidential support for the solutions (Ford, 2012; 
Osborne, 2010). K-12 (Belland et al., 2008; Driver, Newton, & Osborne, 2000; 
Glassner, Weinstock, & Neuman, 2005; McNeill & Pimentel, 2010) and college 
students (Abi-El-Mona & Abd-El-Khalick, 2011; Cho & Jonassen, 2002; Uskola, 
Maguregi, & Jiménez-Aleixandre, 2010) often struggle with argumentation, and 
thus it is important to help them learn this skill. But rather than teaching such didac- 
tically, it is important to put them in a situation about which to argue (Aufschnaiter, 
Erduran, Osborne, & Simon, 2008; Belland et al., 2008; Driver et al., 2000; Jonas- 
sen & Kim, 2010) and support them with such tools as scaffolding (Belland et al., 
2008; Cho & Jonassen, 2002; Clark & Sampson, 2007; Nussbaum, 2002). 


4.2.1.3 Self-Directed Learning Ability 


Self-directed learning refers to the ability to identify learning issues, plan and exe- 
cute a strategy to address the learning issues, and evaluate the quality with which 
the learning issues were addressed; in other words, it is the ability to identify and 
regulate one’s pursuit of learning issues (Bolhuis, 2003; Loyens, Magda, & Rikers, 
2008). Being able to do so is central to addressing ill-structured problems (Giere, 
1990; Jonassen, 2011; Nersessian, 2008), and thus is an important skill to support to 
facilitate student success in problem-centered approaches to instruction (Lohman & 
Finkelstein, 2000; Loyens et al., 2008; Merriénboer & Sluijsmans, 2008). 
Identifying learning issues to be addressed requires that learners assess what in- 
formation is needed to address the problem, and what among the needed knowledge 
is a knowledge deficiency—either not present in the problem presentation or part of 
their preexisting knowledge (Hmelo-Silver, 2004; Loyens et al., 2008). This allows 
for a good deal of autonomy on the part of students in that they can define the content 
to be learned, which in turn can enhance student motivation (Deci & Ryan, 2000; 
Wijnia, Loyens, & Derous, 2011). This clearly goes beyond the traditional practice 
in teacher-centered classrooms in which the teacher determines what is to be learned. 
Planning and executing a strategy to address learning issues requires that learn- 
ers select appropriate learning resources (Hmelo-Silver, 2004; Loyens et al., 2008). 
The effective evaluation of the quality of sources is considered key to information 
literacy and solving problems, as without it, one can be lost in the vast amount of 
information on the web, and not be able to distinguish between credible information 
and non-credible information (Berzonsky & Richardson, 2008; Van de Vord, 2010). 
Yet, college (Berzonsky & Richardson, 2008; Van de Vord, 2010) and K-12 (Kuiper, 
Volman, & Terwel, 2005; Nicolaidou, Kyza, Terzian, Hadjichambis, & Kafouris, 
2011; Williams, 2005) students often experience much difficulty searching for and 
effectively evaluating the quality of online information. For example, K-12 students 
often search for information in an unsystematic manner and rapidly decide if a page 
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is usable; they then quickly search for an answer to a specific question (Kuiper et 
al., 2005). Furthermore, K-12 students often see all evidence as equally valid (Nico- 
laidou et al., 2011). Unaided college science students are often unable to distin- 
guish between peer-reviewed sources and non-peer-reviewed sources (Berzonsky 
& Richardson, 2008). Students’ poor ability to evaluate and use sources effectively 
can stem from such phenomena as conflicting information across sources, complex- 
ity of the target information and the way in which it is portrayed, and the structure 
that the text follows (Britt, Richter, & Rouet, 2014). Unsophisticated epistemic be- 
liefs can cause students to struggle to distill important messages from sources and 
fail to question the credibility of sources (Bråten, Britt, Strømsø, & Rouet, 2011). 
Furthermore, students’ evaluation of sources is often short-circuited by a desire 
for quick learning (Berzonsky & Richardson, 2008; Zimmerman, 1995), which is 
often experienced by students with unsophisticated epistemic beliefs (Chinn, Buck- 
land, & Samarapungavan, 2011; Hofer & Pintrich, 1997; Qian & Alvermann, 1995). 
Clearly, students’ struggles identifying appropriate learning issues and determining 
promising ways to address such present a prime opportunity to use computer-based 
scaffolding (Kuiper et al., 2005). 

The last part of self-directed learning ability is the ability to evaluate the quality 
of one’s own learning and learning processes, also known as metacognition (Loyens 
et al., 2008; Quintana, Zhang, & Krajcik, 2005; Schraw, Crippen, & Hartley, 2006). 
Metacognition is desirable in part to enable the smooth operation and success of a 
student-centered learning environment. This is because if students define and pur- 
sue their own learning issues, and different student groups in the same classroom 
pursue a wide variety of learning issues in a wide variety of manners, it is difficult 
for one teacher to provide sufficient feedback to ensure that all students are on the 
right track. Metacognition can work in concert with teacher feedback to provide a 
consistent corpus of feedback to inform the revision of learning processes as need- 
ed. Metacognition has been an important process that scaffolding seeks to support 
(Cuevas, Fiore, & Oser, 2002; Quintana et al., 2005). 


4.2.1.4 Alignment with NGSS 


The intended learning outcome of promoting higher-order thinking skills aligns 
with NGSS’s emphasis on students learning STEM processes and engaging with 
the culture of STEM and with authentic STEM issues (Achieve, 2013; National 
Science Board, 2010), as detailed in the following sections. 


4.2.1.4.1 STEM Processes 


The goal of helping students learn to apply STEM processes includes helping stu- 
dents learn to (a) identify important problem characteristics to investigate further, 
(b) design strategies to investigate those problem aspects, (c) interpret appropriately 
data and other information collected, (d) arrive at reasonable conclusions, and (e) 
engage in a variety of valued scientific discourse patterns (Achieve, 2013; Duschl, 
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2008; National Science Board, 2010). This does not mean that all citizens need to 
know and be able to apply such processes at the same level as a professional chemist 
or engineer, but they should be able to converse with STEM processes and issues to 
the extent that they can make informed decisions about scientific issues that impact 
their local communities and nation (Duschl, 2008; Kolstø, 2001; Sadler, Barab, & 
Scott, 2007). Each of these subpoints is addressed in the following pages. 


4.2.1.4.1.1 Identify Important Problem Characteristics to Investigate Further 


One of the key processes in STEM is asking cogent questions and identifying key 
aspects of problems (Carr, Bennett, & Strobel, 2012; Giere, 1990; Klahr & Simon, 
1999; National Research Council, 2012; Nersessian, 2008). Going into a problem 
with a vague goal of figuring it out is unlikely to lead to a meaningful solution 
(Jonassen, 2011). Rather, one needs to determine the involved variables, how they 
interact, and what about how they interact is problematic (Belland et al., 2008; 
Jonassen, 2011). This is a key scientific process, and one that does not require the 
asker to be a professional scientist. But it is a skill that individuals do not natu- 
rally have; rather, it needs to be developed through instruction (Jonassen, 2003). By 
habitually asking questions about scientific phenomena, citizens will identify key 
issues facing their communities, and be prepared when others present arguments 
and explanations about STEM-related issues in their community (Kolstø, 2001; 
Sadler et al., 2007; Zeidler, Sadler, Simmons, & Howes, 2005). 


4.2.1.4.1.2 Design Strategies to Investigate Problem Aspects 


Students need to think of scientific problems from different perspectives (Jonassen, 
2011). They also need to recognize and apply the key role of iteration in addressing 
scientific questions (Klahr & Simon, 1999; Nersessian, 2008). Specifically, they 
need to understand that one cannot effectively address a scientific question with just 
one piece of scientific evidence. Rather, they need to collect data/reason scientifi- 
cally in one way, consider the limitations of such, and design and carry out addi- 
tional investigations accordingly (Carr et al., 2012; Giere, 1990; Klahr & Simon, 
1999). In other words, they need to understand STEM from an epistemological 
standpoint—for example, that one cannot arrive at definitive answers to STEM 
questions by consulting just one source or conducting just one investigation (Chinn 
et al., 2011; Duschl, 2008; Hogan & Maglienti, 2001; Mason, Boldrin, & Ariasi, 
2010; Sandoval, 2005) and that most knowledge is not certain (Braten et al., 2011; 
Giere, 1990; Hofer & Pintrich, 1997). But it is not enough to simply understand this; 
citizens need to also be able to and be willing to apply this understanding to real 
STEM problems (Chinn et al., 2011; Mason & Scirica, 2006). 

In designing investigations, students need to be able to apply the tools of math- 
ematics and computation, and recognize the influence of such tools and specifically 
the ways in which the tools are used in the problem solution process (Lesh & Harel, 
2003; National Research Council, 2012; Schoenfeld, 1985). It is important to note 
that applying the tools of mathematics does not simply mean setting up equations. 
Rather, it is important to think, at a conceptual level, about what type of data should 
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be collected and how it will be analyzed to address the research questions (Ker- 
linger & Lee, 2000; Schoenfeld, 1985). This is important so that the right type of 
data is collected. At the same time, students need to understand that not all problem- 
solving strategies need to involve the use of mathematics. Rather, attempting to see 
where the presented problem and an idealized, qualitative model depart from each 
other is a viable problem-solving strategy (Nersessian, 2008). 


4.2.1.4.1.3 Interpret Data and Other Information Appropriately 


Students need to be able to analyze data in a systematic manner, but also realize that 
the job is not done until such analysis is interpreted in light of a theoretical frame- 
work (Giere, 1990; National Research Council, 2012). This is important because 
many individuals have the mistaken impression that scientific investigations always 
take place in a theoretical vacuum. To the contrary, theoretical frameworks always 
drive the design, conduction of, and interpretation of the results of research (Abi- 
El-Mona & Abd-El-Khalick, 2011; Ford, 2012; Giere, 1990; D. Kuhn, 2010). For 
example, theoretical frameworks can influence the choice of problems to investigate 
and the selection of variables on which to focus in an investigation (Lather, 2012; 
Miles & Huberman, 1984). Furthermore, knowing that differences in property A 
are statistically different between two objects means little without interpreting the 
finding in light of a theoretical framework. This is important both as something to 
do when investigating scientific phenomena, but also to remember that other scien- 
tists themselves do this when investigating scientific phenomena (Abi-El-Mona & 
Abd-El-Khalick, 2011; Giere, 1990). 


4.2.1.4.1.4 Arrive at Reasonable Conclusions 


Much of arriving at reasonable conclusions involves interpreting findings in light 
of a theoretical framework (Abi-El-Mona & Abd-El-Khalick, 2011). But it also 
involves actively searching for conflicting findings in the literature. For K-12 stu- 
dents, the literature includes books, interviews with experts, and Internet resources. 
K-12 students need to be able to reconcile conflicting findings to arrive at reason- 
able conclusions. This can involve looking for what the preponderance of studies 
show, privileging findings from more reputable sources, considering limitations and 
delimitations of studies, and synthesizing different elements of findings to create a 
cohesive whole (Britt et al., 2014). This is a challenging activity for such students 
(Braten et al., 2011), who often are blinded by my-side bias (Britt et al., 2014; D. 
Kuhn, 1991; Stanovich & West, 2008). 


4.2.1.4.1.5 Engage in Scientific Discourse Patterns 


Students also need to know and be able to apply and interpret patterns of STEM dis- 
course, including explanations (Britt et al., 2014; Sandoval, 2003) and persuasive 
and dialectical argumentation (Bricker & Bell, 2008; Ford, 2012; Osborne, 2010; 
Perelman & Olbrechts-Tyteca, 1958). Behind all scientific explanations are theo- 
ries, data, and/or biases. Students need to be able to recognize such, both as they 
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create scientific explanations, but also as they interpret those produced by others. 
For example, if a proposal is advanced to dam a river to produce power, citizens 
need to be able to weigh the proposed benefits and drawbacks. Furthermore, they 
need to be able to judge the extent to which an arguer’s stakeholder position influ- 
ences his/her biases, and by consequence, his/her claims and evidence advanced in 
support of his/her position. As part of this process, they need to be able to evaluate 
the credibility of evidence, something with which K-12 and college students often 
struggle (Britt et al., 2014; Nicolaidou et al., 2011). 


4.2.1.4.2 Engaging in the Culture of STEM 


Key to helping students engage in the culture of STEM is helping them learn the 
iterative nature of STEM, as well as the importance of modeling, argumentation, 
and epistemology. 


4.2.1.4.2.1 Iterative Nature of STEM 


Engaging students in the culture of STEM does not mean getting students to engage 
in the “scientific method,” as the latter is in fact heavily simplified (Abd-El-Khal- 
ick, 2012; Lawson, 2010; Tang, Coffey, Elby, & Levin, 2010). STEM profession- 
als do not always start an investigation with a hypothesis, but often engage in an 
exploratory investigation to identify pertinent variables or to simply observe and 
describe a system (Franklin, 2005; Klahr & Simon, 1999; Lawson, 2010). For 
example, exploratory investigations helped scientists uncover the phenomenon of 
gene expression (Franklin, 2005). Such exploratory studies often do not involve a 
control condition, and yet they can lead to very important scientific discoveries, and 
guide further inquiry (Klahr & Simon, 1999). That is, they can indicate and lead 
to descriptions of important phenomena. As such observations accumulate, STEM 
professionals can begin to build theory to explain the phenomena. Further investiga- 
tions can explore whether the new theory explains and predicts other instances of 
similar phenomena (Klahr & Simon, 1999; Lawson, 2010). 

This accumulation of studies along a line of inquiry does not proceed in a linear 
manner. Rather, it proceeds in fits and starts—in a very iterative manner. Students 
should have the opportunity to experience the iterative nature of STEM (T. S. Kuhn, 
1996; Lammi & Becker, 2013; Nersessian, 2008). The iterative nature holds at its 
core theory; theory drives the creation of problem representations (modeling; de- 
scribed below), the design and conduct of investigations to understand problems 
further, the creation of claims, and backing claims with evidence (argumentation; 
described below) (Giere, 1990; Klahr & Simon, 1999; Nersessian, 2008). The ini- 
tial model of a problem situation will necessarily be idealized; it can be improved 
through such processes as establishing limiting cases (Nersessian, 2008), reacting to 
phenomena that cannot be sufficiently explained through existing theory (Klahr & 
Simon, 1999) and engaging with other STEM professionals who often apply differ- 
ent perspectives to problems (Giere, 1990). Not all citizens will engage in the entire 
process of model-building, but they need to understand the process such that they 
can engage in authentic scientific discourses centered on locally relevant scientific 
problems (Kolstø, 2001; Sadler et al., 2007). 


88 4 Intended Learning Outcomes and Assessment of Computer-Based Scaffolding 


Scientists need to revisit theory at multiple stages within the problem-solving 
process, as it can provide a lens through which to view and interpret data, and sug- 
gest new directions to go in an investigation (Giere, 1990; Nersessian, 2008). For 
example, the discovery of the double helix structure of DNA did not occur all at 
once, but rather happened through iteration of ideas and interaction with arguments 
from other scientists (Crick, 1974). Needing to iterate toward an ever-improving 
solution to a scientific problem can be frustrating to students (Belland, Kim, & Han- 
nafin, 2013). Furthermore, students can often see authentic science as consisting of 
only collecting data, and not analyzing such (Gu, Belland, Weiss, Kim, & Piland, 
2015). Thus, it is important to help students control negative emotions and promote 
positive emotions throughout this process (Belland et al., 2013; Kim & Hodges, 
2012; Kim & Pekrun, 2014; Turner & Husman, 2008). But it is also important to 
help students perceive that they can be successful in this endeavor (Bandura, 1977; 
Belland et al., 2013; Britner & Pajares, 2006) and that it is of value (Belland et al., 
2013; Wigfield & Eccles, 2000). 


4.2.1.4.2.2 Modeling 


To be conversant in STEM, individuals also need to be able to use the tools of sci- 
ence, engineering, and mathematics to model natural phenomena, and use those 
models in reasoning and argumentation (Anzai & Yokoyama, 1984; Lesh & Harel, 
2003; Pluta, Chinn, & Duncan, 2011; Sensevy, Tiberghien, Santini, Laubé, & 
Griggs, 2008; Stratford, Krajcik, & Soloway, 1998). This means representing the 
constituent parts of the system and how they interact. This is key to the first part 
of problem-solving—representing the problem (Chi et al., 1981; Jonassen, 2003). 
It is important to be able to model phenomena both qualitatively and also with the 
language of mathematics (Chi et al., 1981; Giere, 1990; Jonassen, 2011; Larkin, 
McDermott, Simon, & Simon, 1980). Modeling phenomena qualitatively means 
thinking widely about the involved entities, using words rather than numbers to 
describe how such entities interact and connecting the problem elements to exist- 
ing domain knowledge (Anzai & Yokoyama, 1984; Jonassen, 2003; Lesh & Harel, 
2003). However, students often suffer from limited understanding of complex cau- 
sality, which can limit their ability to model a problem appropriately (Hmelo-Silver 
& Pfeffer, 2004; Perkins & Grotzer, 2005). That is, one cannot identify a factor A 
that directly causes factor B in all systems; students who think that they should 
always find such a relationship will likely often create an incorrect model (Perkins 
& Grotzer, 2005). 

Students often also suffer from a poor understanding of the words with which to 
precisely describe a scientific relationship; this can lead them to construct represen- 
tations of scientific phenomena that do not reflect reality (Leont’ev, 1974; Sensevy 
et al., 2008). Furthermore, they often perceive that they need to enter values from the 
problem description into an equation, rather than attempt to construct a qualitative 
representation (Van Heuvelen & Zou, 2001). When developing a qualitative model, 
a representation is conducted at first in a learner’s mind, and then can be external- 
ized in such forms as a concept map, a textual representation, and/or a diagram (Chi 
et al., 1981; Jonassen, 2003). The process of articulation can lead to improvement of 
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the model (Belland et al., 2008; Land & Zembal-Saul, 2003; Quintana et al., 2004). 
Qualitative representations can then be iteratively improved. 

Modeling phenomena with mathematics includes setting up an equation that de- 
scribes the phenomena. It is important to note that effective problem solvers do not 
solely model problems qualitatively or quantitatively; rather, they use both sorts of 
representation, as each informs the other and together can lead to a more effective 
solution and solution process (Chi et al., 1981; Jonassen, 2003; Van Heuvelen & 
Zou, 2001). For example, after creating a qualitative model, one may proceed to 
create a quantitative model. The finished qualitative model will influence how the 
quantitative model is set up. One should then see where the models are consistent, 
and where they contradict each other; in this way, the models can be progressively 
improved. By spending adequate time modeling, one can engage in more effec- 
tive problem-solving, as it guides subsequent investigations, can activate solution 
schemas, and can provide the framework by which one can simulate what would 
happen when a variable is manipulated (Anzai & Yokoyama, 1984; Chi et al., 1981; 
Jonassen, 2003; Sins, Savelsbergh, & van Joolingen, 2005). 

Just as it is important to learn to create models, it is also important to be able 
to interpret the models created by others, especially in terms of what these diverse 
models say differently about the underlying problems (diSessa, 1988; Seufert, 
2003; Wu, Krajcik, & Soloway, 2001). Doing so can lead to enhanced understand- 
ing of the problem (Seufert, 2003). This is particularly challenging for K-12 stu- 
dents (Braten et al., 2011; Seufert, 2003). Indeed, learners often simply adhere to 
the model that is closest to their own early experiences, or the simplest explanation 
of the underlying phenomenon, even when presented with a more accurate model 
(diSessa, 1988; Perkins & Grotzer, 2005). This may be explained in part by most 
K-12 students’ lack of familiarity with complex causal models, such as those that 
explain changes in a factor through indirect action from a combination of factors A 
and B (Perkins & Grotzer, 2005). While some evidence indicates that reluctance to 
consider an alternative model is widespread among learners of differing levels of 
prior knowledge and skill, other evidence indicates that it may be more prevalent 
among lower-achieving students (Seufert, 2003). Thus, it is especially important 
to endeavor to increase modeling skills from a social justice vantage point and to 
broaden participation in STEM (Lynch, 2001). 


4.2.1.4.2.3 Argumentation 


Science is very much a social endeavor, as no scientist works in a vacuum (Ford, 2012). 
Rather, scientists work in a large community of practice in which they share and defend 
findings to one another, and build off of others’ work. At the core of this is argumenta- 
tion, defined as both backing claims with evidence and models, but also effectively eval- 
uating claims on the basis of evidence and models (Ford, 2012; Osborne, 2010). The 
argumentation process allows scientific models and theories to be iteratively improved 
(Ford, 2012). To be able to engage in STEM effectively as citizens, individuals also 
need to be able to engage in clear argumentation (Aufschnaiter et al., 2008; Jonassen, 
2011; Osborne, 2010; Perelman & Olbrechts-Tyteca, 1958). For example, when scien- 
tific issues are discussed, citizens need to be able to sort out well-founded claims from 
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less-well-founded claims. K-12 students (Hogan & Maglienti, 2001; Weinstock, Neu- 
man, & Tabak, 2004) and adults (D. Kuhn, 1991) often struggle to evaluate arguments, 
in part due to poor ability to evaluate the credibility of evidence (Braten et al., 2011; 
Nicolaidou et al., 2011). 

There are several key areas that need to be addressed in the course of learning 
to argue. First, there is the conceptual level—helping students understand what a 
well-founded argument is and is not, and by extension recognize strong and weak 
arguments. After all, before one can hope to help students learn a skill, they need 
to be familiar at a conceptual level with the skill that is being learned (Wood et al., 
1976). Specifically, students need to understand that an argument is linking a claim 
to evidence by way of premises to which the claimer and the audience adhere, in the 
pursuit of leading the audience to adhere to the claim (Perelman & Olbrechts-Tyte- 
ca, 1958). A well-founded argument is one that performs this function well, within 
the framework of generally accepted rhetorical principles. Being able to distinguish 
between strong and weak arguments relies in part on sophisticated epistemological 
understanding (Hogan & Maglienti, 2001; Weinstock et al., 2004), which refers 
to how one thinks that knowledge is established and justified (Mason & Scirica, 
2006). This is described in more detail in the next section. 

Next, individuals need to learn about the process of argumentation. This involves 
first making a claim. But before one can establish a claim, one needs to thoroughly 
understand the underlying problem, including the involved entities and how they in- 
terrelate. To do so, one needs to define the problem, determine needed information, 
and find and organize the information (Belland et al., 2008). Next, one needs to con- 
nect evidence to the claim. In so doing, one needs to appeal to premises by which 
the evidence connects to the claims. Ideally, one employs premises with which the 
audience already agrees (Perelman & Olbrechts-Tyteca, 1958). Premises that are 
widely held by the majority of the audience can be left unsaid, while premises that 
are not held as given by the majority of the audience need to be stated (Perelman 
& Olbrechts-Tyteca, 1958). For example, if one wanted to claim that Brazilians 
are unhappy that the Brazilian team was knocked out of the World Cup, one could 
provide evidence that the Brazilian team in fact was knocked out of the World Cup 
and that many Brazilians are unhappy. One would also rely on a premise that people 
tend to be unhappy when their national team in their most popular sport loses. 


4.2.1.4.2.4 Epistemology 


Closely connected to learning argumentation is a need to develop sophisticated 
epistemic beliefs, defined as beliefs about the sources, certainty, justification, 
and simplicity of knowledge that align with that of most STEM professionals 
(Bendixen & Rule, 2004; Hofer & Pintrich, 1997). With sophisticated epistemic 
beliefs, an individual knows that claims need to be supported with well-justified, 
converging evidence, such as evidence collected through tests of a refutable 
question (Chinn et al., 2011; Hogan & Maglienti, 2001; Mason & Scirica, 2006; 
Weinstock et al., 2004). Next, with sophisticated epistemic beliefs, one understands 
that justification for knowledge claims should come from rational arguments or 
empirical evidence, rather than an appeal to authority (Hogan & Maglienti, 2001; 
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Jiménez-Aleixandre, 2014). Furthermore, with sophisticated epistemic beliefs, one 
understands that arriving at correct information/conclusions will often not hap- 
pen instantaneously (Chinn et al., 2011; Greene, Azevedo, & Torney-Purta, 2008). 
Someone with sophisticated epistemic beliefs will also understand that most knowl- 
edge is not certain, and rather is subject to verification through further research 
(Hofer & Pintrich, 1997). Without sophisticated epistemic beliefs, individuals often 
jump to erroneous conclusions (Hofer, 2001; Weinstock et al., 2004). Epistemic 
beliefs influence individuals’ ability to interpret conflicting information from mul- 
tiple scientific texts (Bråten et al., 2011). The sophistication of middle school stu- 
dents’ epistemic beliefs significantly predicted their ability to produce arguments, 
counter-arguments, and rebuttals (Mason & Scirica, 2006). Epistemic beliefs have 
also been associated with conceptual change: the more sophisticated the epistemic 
beliefs, the easier it is to achieve conceptual change given the proper instruction, 
and vice versa (Hofer, 2001). 


4.2.1.4.3 Engaging with Authentic STEM Issues 


To be clear, the idea of helping all citizens learn some cross-cutting concepts does not 
mean reestablishing a rhetoric of conclusions approach to science education (Chinn 
& Malhotra, 2002; Duschl, 2008)—one focused on transmitting an unchanging body 
of scientific knowledge. Rather, it means to teach core concepts in science for which 
evidence is overwhelming, such as the role of DNA and genetic expression in deter- 
mining such characteristics as the size, shape, and function of organisms. One can 
do this by engaging students with authentic STEM problems. Authentic problems are 
characterized by the following factors: they (a) are locally relevant, (b) have multiple 
valid solutions and solution paths, and (c) relate to one or more aspects of STEM, 
and addressing them requires the use of the tools of the discipline (Barab, Squire, & 
Dueber, 2000; Chinn & Malhotra, 2002; Hung & Chen, 2007; Jonassen, 2011). 

Authentic problems suitable for use in STEM education include (a) dilemmas, 
a problem type represented by many socioscientific issues, and (b) design prob- 
lems, which may be centered in or at least involve engineering education (Jonassen, 
2000). A socioscientific dilemma can address whether a factory should be built that 
would cause pollution and degrade habitat, but would increase jobs (Tal & Kedmi, 
2006). To address this problem, students need to consider such scientific concepts 
as what contributes to the health or lack thereof of coastal habitats. But they also 
need to consider social equity issues related to the right to work in an appropriate 
job. Many such problems can involve multiple areas within STEM, as interdis- 
ciplinary work can lead to more robust problem solutions (Belland & Fee, 2012; 
Porter & Rafols, 2009) and is becoming more common in STEM research (Murray, 
Atkinson, Gilbert, & Kruchten, 2014; Porter & Rafols, 2009). 

A design problem could involve how to use design to prevent erosion while 
supporting local habitat on barrier islands (Kolodner et al., 2003). To address this 
problem, middle school students need to employ engineering design principles and 
processes, draw on scientific knowledge, identify and research needed knowledge, 
and engage in extensive iteration. This engages students in the culture of STEM, 
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but also helps them gain important STEM skills and knowledge. Another design 
problem could involve the design of an alarm to respond to specific needs (Silk, Sc- 
hunn, & Cary, 2009). Addressing this problem again requires the use of engineering 
approaches and scientific knowledge. 

Requiring the use of the tools of the discipline means that students should need to 
engage in similar processes and use similar tools as professionals in the target field 
(Chinn & Malhotra, 2002; Hung & Chen, 2007). It is clear that no students except 
the most advanced graduate students will use exactly the same processes and tools 
as professional scientists and engineers, but they should use similar epistemic pro- 
cesses, defined as approaches to designing and conducting investigations, as well as 
interpreting data and making conclusions (Chinn & Malhotra, 2002). 


4.2.2 Learning Content Deeply 


Learning content deeply goes beyond simple declarative learning; rather, it refers to 
the ability to describe knowledge in one’s own words and apply it to new situations, 
as well as recognize the connections between the knowledge and related knowledge 
(Belland et al., 2009; Bloom, Englehart, Furst, Hill, & Krathwohl, 1956). This out- 
come has been the focus on much work in scaffolding. One line of such research is 
that of knowledge integration (Clark & Linn, 2013; Linn, 2000). According to this 
framework, the knowledge learners bring to school does not need to be replaced by 
more accurate models, but rather can be used as a base on which to build greater 
understanding. This is because students’ existing knowledge base about science 
consists of mini theories developed through experience that may be at least partially 
correct (diSessa, 1988). One can help students build upon their existing knowledge 
base by encouraging them to engage in authentic problem-solving scenarios sup- 
ported by scaffolds. However, the goal is not directly to improve problem-solving 
ability. Rather, it is to help students (a) build enhanced mental models of such things 
as natural phenomena, and (b) realize that what they are learning applies equally 
well at home and out in the world as in school (Clark & Linn, 2013; Linn, 2000). 
However, there is the thought that this in turn could lead to more effective problem- 
solving (Linn, 2000). 

Another line of research on scaffolding that focuses on deep content learning 
is that of intelligent tutoring systems. In this context, learning content deeply has 
a different meaning than in scaffolding to support knowledge integration. Name- 
ly, intelligent tutoring systems seek to develop students’ procedural (production 
rules) and declarative knowledge related to a particular skill (Anderson, Matessa, 
& Lebiere, 1997; Self, 1998; VanLehn, 2011). Scaffolding embedded in intelligent 
tutoring systems helps students apply declarative knowledge to problems. In this 
way, students develop production rules by which the declarative knowledge can be 
applied without conscious control to similar problems in the future (Koedinger & 
Aleven, 2007). But Adaptive Character of Thought-Rational (ACT-R) also endeav- 
ors to help students learn declarative knowledge deeply, which means that it can be 
deployed independently in the future. 
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The NGSS and Common Core posit learning content deeply as an important goal 
(Achieve, 2013; McLaughlin & Overturf, 2012; National Science Board, 2010). For 
example, one part of the NGSS calls for students to learn cross-cutting concepts. 
Cross-cutting concepts takes at its core the idea that certain concepts—“patterns; 
cause and effect: mechanism and explanation; scale, proportion, and quantity; 
systems and system models; energy and matter; flows, cycles, and conservation; 
structure and function; and stability and change’”—are applicable across a range 
of STEM disciplines (National Research Council, 2012, p. 3). For example, cause 
and effect applies equally in science and engineering, and indeed among the many 
subdisciplines in science and engineering. It is important to note that one cannot 
always find a single cause that by itself leads to a given effect; often there are 
multiple causal factors that either together lead to the given effect, or which moder- 
ate each other’s effect (Hmelo-Silver, Marathe, & Liu, 2007; Perkins & Grotzer, 
2005). Seeking to find causal factors for phenomena is a core activity in science 
(Achieve, 2013) and engineering (Brophy, Klein, Portsmore, & Rogers, 2008; Carr 
et al., 2012). Furthermore, in engineering, one most often aims to design a product, 
tool, or strategy that causes a desired outcome (National Research Council, 2012). 
Considering scale, quantity and proportion is just as important in physics as it is in 
chemistry, and indeed is important in mechanical and other forms of engineering. 

Such cross-cutting concepts are key to the participation of common citizens in 
discourses about STEM problems. For example, without knowing about flows and 
cycles as well as systems, one would not be able to intelligently discuss issues 
related to water quality and access. It is unreasonable to expect everyone to take 
environmental science classes to learn about such concepts within the context of 
water quality, and chemistry classes to learn about the application of such concepts 
in chemistry, and so on. Rather, the hope is that students can learn the concept as 
a cross-cutting concept in one context, and add depth to their knowledge when 
learning the same cross-cutting concept in another context, as in a spiral curricu- 
lum (Achieve, 2013; Bruner, 2009). Or, at the very least, they would have the base 
knowledge so that when an authentic socioscientific issue arises, they would be 
able to converse with it intelligently (Reiser, Krajcik, Gouvea, & Pellegrino, 2014). 

Cross-cutting concepts may be best learned in the context of problem-centered 
instructional models (National Research Council, 2007, 2012). However, abstract- 
ing a generalizable cross-cutting concept from such a problem is not easy (Perkins 
& Grotzer, 2005). First, the target concept may be experienced as context-specific 
by the student (Perkins & Salomon, 1989). Next, it is not an easy feat to both encode 
such a concept and include the necessary information to be able to retrieve it later 
in a new situation in which the concept could be applied (Perkins & Grotzer, 2005). 
Thus, one may need to be explicit about the cross-cutting nature of concepts, as well 
as situations in which they can be applied in the future, though this does not need to 
be done in a didactic manner. 

The NGSS also call for students to learn disciplinary core ideas, defined as a few 
key ideas in each STEM discipline around which one can build STEM curricula 
(Achieve, 2013; National Research Council, 2012). For example, a core idea in 
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physical sciences revolves around the structure and properties of matter (National 
Research Council, 2012). A core idea in life sciences relates to the growth and de- 
velopment of organisms (National Research Council, 2012). This approach reflects 
in many ways the idea of science from a few ideas—the idea that it is more impor- 
tant to know very well a few core ideas in a scientific field, rather than know less 
well a wide breadth of topics in the given science discipline (Clark, 2000; Pritchard, 
Barrantes, & Belland, 2009; Schmidt, Wang, & McKnight, 2005). The six countries 
that performed the best in the Third International Mathematics and Science Study 
(TIMSS) focused on a much narrower range of key science concepts than most 
states/districts in the USA (Schmidt et al., 2005). Understanding core ideas does not 
mean simply being able to describe the idea, but rather to use the idea to describe 
natural phenomena (Bloom et al., 1956; Reiser et al., 2014). This aligns with the 
focus on deep content learning of much scaffolding (Clark & Linn, 2013; Linn, 
Clark, & Slotta, 2003). 


4.2.3 Results from Meta-Analysis 


In the meta-analysis, outcomes were coded according to whether scaffolding in the 
studies aimed to increase higher-order thinking skills (7 icome, 23 /)» content learn- 
ing (1, comes 795), OF motivation (71, romes= 13 See Table 4.1; Belland et al., In Press). 
This means that 71.2% of included outcomes aimed at enhancing higher- 
order skills, 28.5% aimed at enhancing content knowledge, and 0.3% aimed to 
enhance motivation. Results indicated that there was no statistically significant dif- 
ference between average effect sizes when scaffolding intended to increase higher- 
order thinking skills (g=0.45) versus deep content learning (g=0.50). This sug- 
gests that scaffolding is a robust instructional approach that can be used to promote 
diverse learning goals. This is interesting, in that educational interventions tend to 
not have equally positive influences on content learning and higher-order skills. For 
example, lecture is well known to be efficient and effective at influencing content 
learning, but to be ineffective at influencing higher-order thinking abilities (Alba- 
nese & Mitchell, 1993; Bland, Saunders, & Frisch, 2007). Problem-based learning 
tends to lead to strong impacts on higher-order thinking skills, and not on immedi- 
ate recall of content (Gijbels, Dochy, Van den Bossche, & Segers, 2005; Walker 
& Leary, 2009). Thus, scaffolding appears to remedy one of the weaknesses of 
problem-based learning, by helping students learn content knowledge effectively. 


Table 4.1 Table of results of moderator analyses on the effect of intended learning outcome on 
cognitive outcomes 


| | 95% confidence interval 


Level n outcomes | Effect size estimate | Lower limit | Upper limit 
Content learning | 95 [0.50 [0.41 [0.58 
‘Enhance motivation | 1 | 0.86 0.2 [1.52 
‘Higher-order thinking | 237 [0.45 [0.39 [0.51 
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Scaffold designers can set out to design scaffolds with the intention of enhancing 
students’ higher-order thinking abilities or content knowledge. But to be able to 
verify if the scaffolding that is produced actually enhances such knowledge and 
skills, it is necessary to consider how the learning is assessed (Cronbach, 1949; 
Messick, 1989). After all, an assessment that is on the topic of problem-solving 
does not necessarily assess problem-solving ability. To assess problem-solving, one 
would need to assess students’ abilities to define the problem, determine needed 
information, and find and synthesize the needed information to arrive at a solution 
(Belland et al., 2009; Sugrue, 1995). 

To assess learning appropriately, it is important to consider the constructs of 
interest, defined as a characteristic of an individual or group (e.g., intelligence, flu- 
ency, and argumentation ability) that cannot be directly measured, and for which 
one can only measure certain related behaviors (e.g., ability to construct an argu- 
ment given a scenario and argument construction parameters) (Belland et al., 2009; 
Kerlinger & Lee, 2000; Messick, 1989). It is necessary to carefully define the con- 
structs to be assessed, and craft a set of activities that can reliably and validly assess 
the extent to which the test takers evidence a grasp of the target construct (Anastasi 
& Urbina, 1997; Belland, 2012; Belland et al., 2009; Cronbach, 1949; Messick, 
1989). To be reliable, test scores need to be consistent when taken multiple times 
in close temporal proximity by the same person and also display similar response 
patterns among people of similar abilities (Kerlinger & Lee, 2000; Messick, 1989). 
To be valid, a variety of evidence needs to support the conclusion that the set of test 
scores issuing from the administration of a test are a fair reflection of the amount of 
the underlying construct the test taker has (Kerlinger & Lee, 2000; Messick, 1989). 
To be valid, a set of test scores needs to also be reliable (Messick, 1989). 

When examining assessment of learning results from the use of computer-based 
scaffolding, it is useful to consider the assessment framework of Sugrue (1995), who 
classified assessments in terms of whether they measure at the concept, principles, 
or application level (see Fig. 4.2). When doing so, it is important to avoid the temp- 
tation to label all multiple choice assessments as concept-level assessments, and 
all open-response assessments as principles or application-level assessments (Han- 
cock, 1994). Measuring at the concept level means that the assessment measures 
how well students can define or recognize examples of a given concept. This could 
include assessments ranging from multiple choice tests in which students need to 
choose a definition, to sorting tasks, and short answer assessments. Measuring at 
the principles level means that students are provided scenarios involving relation- 
ships among several variables and need to predict what would happen if one of 
the variables were manipulated in a particular way. This again could take many 
different forms, ranging from multiple choice to writing essays. Measuring at the 
application level means that students need to design and conduct an investigation 
using the newly learned material. This is often a performance-based assessment, but 
can take other forms, such as multiple choice (Hancock, 1994). In many ways, the 
concept, principles, and application levels parallel the intended learning outcomes 
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Assessment 
Levels 


Concept Principles Application 


Aim of assessment Aim of assessment Aim of assessment 
level: level: level: 
Determine if students Determine if, givena Determine if students 
can define or recognize relationship among can design and 
examples of given several variables, conduct a new 
concept students can predict investigation using 
what will happen when newly learned 
one variable is material. 
manipulated. 


Fig. 4.2 Aims of assessments at the concept, principles, and application levels, as proposed by 
Sugrue (1995) 


of scaffolding. But it is important to make the distinction between intended learning 
outcomes and assessment levels, as the former are goals towards which designers 
work when designing scaffolds, and the latter are the ways in which student learn- 
ing is assessed. These are not always one and the same (Boud & Falchikov, 2006). 


4.3.1 Results from Meta-Analysis 


It is natural to question whether there are any differences in effect sizes of comput- 
er-based scaffolding according to the different assessment levels. For example, if 
scaffolding is designed to promote problem-solving ability, one would imagine that 
assessment at the principles or application levels would be more sensitive to the ef- 
fect of said scaffolding. And if scaffolding is intended to influence content learning, 
then one would expect that concept-level assessment would be most sensitive to 
the effect of the scaffolding. Outcomes at the concept level (1, .4mes= 125), prin- 
ciples level (17,,, ames 167), and application level (715, jomes = 41) were included (See 
Table 4.2; Belland, Walker, Kim and Lefler, In Press). Scaffolding’s impact 
on cognitive outcomes was statistically greater when measured at the principles 
level (g=0.51) than when measured at the concept level (g=0.40). The effect size 
for scaffolding at the application level was g=0.44. Thus, the effect size point esti- 
mate for scaffolding ranged from 0.40 to 0.51 for the three assessment levels. The 
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Table 4.2 Table of results of moderator analyses on the effect of assessment level on cognitive 
outcomes 
| 95 % confidence interval 


Level |n outcomes Effect size estimate | Lower limit | Upper limit 
Concept | 125 | 0.40 | 0.33 | 0.47 
Principles | 167 0.51 0.44 (0.59 
Application | 41 [0.44 (0.32 | 0.57 


95% confidence intervals—(0.33-0.47), (0.44—0.59), and (0.32—0.57) for concept, 
principles, and application level assessment, respectively—indicate that one can 
have great confidence that scaffolding leads to substantial effects across all three 
assessment levels. This is intriguing, in that it is rare for educational interventions to 
have such a consistent effect across assessment levels. For example, the underlying 
instructional models with which scaffolding is used often produce strong effects in 
one or two of the assessment levels, but not all three. Problem-based learning (PBL) 
meta-analyses have indicated the PBL leads to effects that are statistically greater 
than zero at the principles (Gijbels et al., 2005) or the principles and application 
levels (Walker & Leary, 2009), but not at the remainder of the assessment levels. 

There are several possible explanations of the robust effect of scaffolding across 
assessment levels. First, scaffolding designed to impact higher-order thinking abili- 
ties may only be assessed at the principles and application levels, and be mostly 
successful at influencing student learning as measured by the given assessments; 
likewise, scaffolding designed to influence content learning may be assessed largely 
at the concept level, and be mostly successful in influencing learning at that level. 
Next, it may be possible that scaffolding designed to enhance content learning is 
also assessed at the principles and application levels, and it also has a positive influ- 
ence at those levels. It is possible also that scaffolding designed to enhance higher- 
order thinking abilities is assessed at the concept, principles, and application levels, 
and leads to strong learning outcomes at all three levels. After all, one of the argu- 
ments for promoting content learning in the context of problem-solving is that this 
will increase students’ abilities to solve problems through the enhancement of stu- 
dents’ mental models (Anderson, 1983; Clark & Linn, 2013; Johnson-Laird, 2001). 

It is especially interesting that scaffolding leads to such a strong effect at the ap- 
plication level. The lower limit of its confidence interval was 0.32, which is an effect 
of a substantial magnitude—one that is higher than one often finds in educational 
technology applications for mathematics learning (ES=0.15; Cheung & Slavin, 
2013). To perform well on an application level assessment, one must understand the 
target strategy to a sufficient extent to be able to apply it to a new situation (Sugrue, 
1995). This is a very difficult bar to clear, as it requires abstraction of the underlying 
strategy, and application of said strategy in a new situation that likely differs in key 
aspects. In short, it is essentially far transfer that is being targeted, which is very 
difficult to promote (Barnett & Ceci, 2002; Salomon & Perkins, 1989). 
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Chapter 5 
Computer-Based Scaffolding Strategy 


Abstract This chapter covers variations in scaffolding strategies along the follow- 
ing characteristics—scaffolding function (e.g., strategic and conceptual), context 
specificity (i.e., generic or context-specific), customization (e.g., fading and fading/ 
adding), and customization schedule (e.g., self-selected and performance-based). 
These variations and the theoretical basis for these are explained. Then, results from 
the meta-analysis are shared, which indicate that there are no differences in cogni- 
tive outcomes according to scaffolding function, context specificity, and customiza- 
tion. These results are then discussed. 


Keywords Adding - Conceptual scaffolding - Context-specific - Fading - Fading/ 
adding - Fixed customization - Generic - Metacognitive scaffolding - Modeling - 
Motivation scaffolding - Performance-based customization - Question prompts - 
Strategic scaffolding - Scaffolding customization - Self-selected customization 


5.1 Rationale for Chapter 


There is a very large literature on what computer-based scaffolding should do and 
why, including conceptual frameworks (e.g., Belland, Kim, & Hannafin, 2013; 
Quintana et al., 2004) and guidelines derived from empirical studies (e.g., Lee & 
Songer, 2003). While these articles and other reports are often well referenced, by 
necessity, they only draw on some of the empirical studies/evidence on computer- 
based scaffolding as well as theoretical analysis. Furthermore, their messages about 
what forms of scaffolding are most effective are often conflicting. As such, it is 
difficult for scaffolding designers and researchers to know what scaffolding ap- 
proaches are most effective under what circumstances. 

A key goal of the meta-analysis that I completed with my colleagues was to syn- 
thesize empirical evidence on scaffolding so as to uncover the most effective scaf- 
folding strategies in science, technology, engineering, and mathematics (STEM) 
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education. This way, scaffolding designers and researchers could have solid, em- 
pirically based rationales for using one strategy over another in a particular context. 
And in the case that variation of a scaffolding characteristic did not influence cogni- 
tive outcomes, designers could be relatively confident that their choice would not 
adversely affect student learning one way or the other’. 

To accomplish this, it was first important to think about ways in which scaffold- 
ing strategies can vary. The first such way is the scaffolding function, defined as 
the focus of its support (Hannafin, Land, & Oliver, 1999). This is different from 
scaffolding’s intended outcome in that scaffolding function focuses on the areas in 
which scaffolding needs to assist students so as to facilitate student success at the 
target task. Scaffolding function can be categorized into conceptual scaffolding, 
strategic scaffolding, metacognitive scaffolding, and motivation scaffolding (Bel- 
land, Kim, et al., 2013; Hannafin et al., 1999), each of which is described in the 
sections that follow, along with meta-analysis results on the relative influence of 
each type of scaffolding on cognitive outcomes. 

Next, one can consider scaffolding in terms of whether it contains embedded 
content knowledge (context-specific) or not (generic) (Belland, Gu, Armbrust, & 
Cook, 2013; Davis, 2003; McNeill & Krajcik, 2009). This has to do with whether 
context-specific information is embedded in the scaffolding support. For example, 
consider a scaffold that helps students consider where to build a power plant. A 
generic version of the scaffold may provide a generic process by which individuals 
can (a) identify needed characteristics of a site for an industrial building, (b) iden- 
tify locations that have at least some of those characteristics, (c) list pros and cons 
of the different identified sites, and (d) select a site and build a rationale for why the 
site is appropriate. A context-specific version may be tailored entirely to the choice 
of a location for a power plant, and all prompts would be couched in that context. 
Furthermore, a context-specific scaffold may include the options from which stu- 
dents can choose as well as the information with which students will make their de- 
cision. Decisions to embed such information are often based on theories of whether 
target skills are context-specific or generic (Davis, 2003), a question on which there 
is much disagreement (Perkins & Salomon, 1989). In the following sections, this 
scaffolding characteristic is explained, along with the influence of each level of this 
variable on scaffolding’s influence on cognitive outcomes. 

Finally, one can consider scaffolding in terms of how it is (or is not) custom- 
ized. Customization can include fading, adding, fading/adding, or none, and can be 
done on the basis of performance, self-selection, a fixed schedule, or none (Collins, 
Brown, & Newman, 1989; Koedinger & Corbett, 2006; Pea, 2004). 


' Note: One needs to consider results of meta-analyses alongside results of other research, espe- 
cially qualitative and other research that would be excluded from meta-analyses. Also, it is impor- 
tant to note that the current meta-analysis only included cognitive outcomes. Other outcomes such 
as motivational ones are also important in a holistic assessment of the influence of an instructional 
approach. 
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5.2 Scaffolding Function 


Scaffolding functions include conceptual scaffolding, strategic scaffolding, meta- 
cognitive scaffolding, and motivation scaffolding (Belland, Kim, et al., 2013; Han- 
nafin et al., 1999). These are detailed in the following sections, and results from the 
meta-analysis comparing the effectiveness of such functions are presented. 


5.2.1 Conceptual Scaffolding 


Conceptual scaffolding guides students in terms of things to consider when solving 
problems (Hannafin et al., 1999; Sandoval & Reiser, 2004). In any problem, there 
are a multitude of possible things to consider when solving it, and thus it is impor- 
tant to help students narrow these down and choose more productive considerations 
(Jonassen, 2000) and make sense of the data and information encountered (Ford, 
2012; Quintana et al., 2004). Such scaffolding can take a more structured approach 
when informed by ACT-R (Anderson, Matessa, & Lebiere, 1997) or knowledge in- 
tegration (Linn, Clark, & Slotta, 2003), or a less structured approach when informed 
by activity theory (Belland & Drake, 2013; Luria, 1976). In computer-based scaf- 
folding, conceptual scaffolding can take the form of expert modeling in which an 
expert discusses what aspects of a problem he/she would consider in the process of 
addressing a problem (D. D. Li & Lim, 2008; Pedersen & Liu, 2002). For example, 
in Alien Rescue, an expert discussed what considerations he would make when con- 
sidering what planet to choose as a new home for a stranded alien (Pedersen & Liu, 
2002). This led experimental students to develop significantly stronger rationales 
for their problem solutions and to be less likely to ask vague questions than control 
students (Pedersen & Liu, 2002). Expert modeling would likely be seen more often 
in scaffolding informed by activity theory than in scaffolding informed by ACT-R 
or knowledge integration. 

Conceptual scaffolds can also invite students to plan animations or experiments, 
directing them to areas of planning that are particularly important and to which stu- 
dents should pay great attention, and simplifying areas that are not central to learn- 
ing goals (Reiser, 2004). For example, a scaffold invited students to plan a chemi- 
cal reaction animation they would create, create the animation in a modeling tool, 
explain the meaning of the animation and relate it to the phenomenon it describes, 
and evaluate it (Chang, Quintana, & Krajcik, 2009). Engaging in the full process led 
students to perform better on a test of chemistry achievement, as well as animation 
and interpretation quality, as compared to students who either just designed and 
created the animation, or designed, created, and interpreted the animation (Chang et 
al., 2009). In another example, students can use a simulation to model the behavior 
of ions near a cell membrane (Nichols, Hanan, & Ranasinghe, 2013). Students can 
modify the number of potassium or sodium channels and see how the simulation 
responds, and they also read prompting questions that indicate important elements 


110 5 Computer-Based Scaffolding Strategy 


to consider (Nichols et al., 2013). It was found that experimental students engaged 
in richer collaborative discussions and evidenced less misconceptions on a posttest 
than control students (Nichols et al., 2013) 

Conceptual scaffolds can also use such tools as concept mapping to list impor- 
tant concepts in the material being learned and invite students to make connec- 
tions between such concepts explicit through the use of connecting arrows (Chin, 
Dohmen, & Schwartz, 2013). Then, pedagogical agents (teachable agents) are 
asked questions, and the veracity of their answers depends on the appropriateness 
of the connections made in the concept map (Chin et al., 2013). This approach led 
experimental students to perform significantly better on tests of content knowledge 
and the ability to organize explanations according to categories (e.g., carnivore vs. 
herbivore) (Chin et al., 2013). In another example, Belvedere invited high school 
students to create claims, evidence elements, and premises, and to make connec- 
tions among the different elements to create an evidence-based argument (Toth, 
Suthers, & Lesgold, 2002). Students who were invited to engage in concept map- 
ping and to reflect on their work performed significantly better in overall reasoning 
than students who engaged in mapping without reflection, as well as students who 
wrote prose with and without reflection (Toth et al., 2002). In another example, 
elementary students conducting web-based inquiry were given a concept mapping 
tool along with guidance on how to link different concepts they encountered/were 
learning, and also guidance for searching and presentation design (MacGregor & 
Lou, 2004). Students who used the scaffolding recalled significantly more content 
from the investigation and also had significantly more creative and organized final 
presentations (MacGregor & Lou, 2004). 


5.2.2 Strategic Scaffolding 


Strategic scaffolding bootstraps a strategy that students can use to solve a problem 
(Hannafin et al., 1999; Reiser et al., 2001). From an activity theory perspective, this 
approach would still leave open the possibility for student agency in the applica- 
tion of the strategy, and possible modification thereof. This is because according 
to this framework, the semiotic process of building signs according to tools (e.g., 
scaffolds) is highlighted (Belland & Drake, 2013; Wertsch & Kazak, 2005). For ex- 
ample, a scaffold bootstrapped positive collaboration skills by providing a database 
of positive groupwork rules, inviting students to (a) create their own groupwork 
rules, (b) evaluate their group processes in light of the group rules they created, (c) 
discuss according to given discussion questions, and (d) self-evaluate the whole 
process (Ulicsak, 2004). Experimental students engaged in more lengthy discus- 
sions and exhibited greater reflection (Ulicsak, 2004). As another example, the 
Connection Log leads middle school students through a generic argument creation 
process (Belland, 2010) grounded in the persuasive theory of argumentation (Perel- 
man & Olbrechts-Tyteca, 1958). This led lower-achieving experimental students 
(Belland, Glazewski, & Richardson, 2011) and average-achieving experimental 
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students (Belland, 2010) to evaluate arguments significantly better than their con- 
trol counterparts. 

From an ACT-R perspective, the possibility for choice in the application of the 
strategy would be limited due to the desire to minimize unsuccessful practice (An- 
derson, 1983; Koedinger & Corbett, 2006). For example, in an intelligent tutoring 
system designed to help students learn LISP programming, the system provides a 
LISP programming task for students to do and a template for programming ele- 
ments that need to be in the program (Corbett & Anderson, 2001). Students can type 
programming commands in a window, and the system checks the code and provides 
either immediate feedback, error flagging, or self-selected feedback (Corbett & An- 
derson, 2001). Such feedback was designed so as to promote speed in reaching the 
correct answers, consistent with the assumption in ACT-R that struggle is not desir- 
able (Anderson et al., 1997). Students who received feedback made significantly 
fewer errors on the posttest than students who did not receive feedback (Corbett & 
Anderson, 2001). 

From a knowledge integration perspective, choice may be allowed to the extent 
to which students’ existing problem-solving schemas would be elicited and com- 
pared to provided normative strategies (Linn et al., 2003). But at the same time, 
allowing for student choice in the use of strategies is not an overt goal from the 
knowledge integration perspective in that the existence of normative strategies is 
posited. 


5.2.3 Metacognitive Scaffolding 


Metacognitive scaffolds invite and help students to evaluate their own thinking 
(Cuevas, Fiore, & Oser, 2002; Hannafin et al., 1999). Within scientific inquiry, im- 
portant metacognitive processes include task definition and planning, monitoring 
and regulating, and reflection (Quintana, Zhang, & Krajcik, 2005). Metacognitive 
scaffolding can help students with several areas of the metacognitive process, in- 
cluding planning, monitoring and regulating, and reflection (Quintana et al., 2005). 
Metacognitive scaffolding focused on planning gives students tools for planning 
and also prompts them to consider the importance of the planning process (Aze- 
vedo, 2005; Quintana et al., 2005). Metacognitive scaffolding to enhance monitor- 
ing and regulating can focus on monitoring one’s progress through the inquiry task 
according to a set of mileposts (Cuevas et al., 2002; Zhang & Quintana, 2012). 
Metacognitive scaffolding to enhance reflection can invite students to evaluate 
the quality of ideas and products generated according to rubrics (Cuevas et al., 
2002; Quintana et al., 2005). For example, this may be by giving students criteria 
to make the evaluation and a forum in which to do so. A metacognitive scaffold 
invited middle school mathematics students to respond to questions emphasizing 
comprehension, connection, strategy, and reflection (Kramarski & Hirsch, 2003). 
Students who used the metacognitive scaffolding performed significantly better on 
a posttest of algebraic thinking than control students (Kramarski & Hirsch, 2003). 
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In another example, university students were given prompts encouraging them to 
stop and reflect on their answers to two questions regarding human immune sys- 
tems, and a concept map they made with the pertinent concepts (Ifenthaler, 2012). 
These prompts were either generic or context-specific; students who received the 
generic prompts gained significantly more from pre- to posttest of domain-specific 
knowledge than students who received context-specific prompts and those in the 
control group (Ifenthaler, 2012). 

Metacognitive scaffolds are not universally effective. For example, a metacogni- 
tive scaffold contained three tools to help college students during a computer lit- 
eracy test—a project planning sheet, a tool to make connections in information, and 
a project reflection sheet (Su & Klein, 2010). Students who received metacognitive 
scaffolds performed significantly worse on a posttest than students who received 
conceptual scaffolds (Su & Klein, 2010). In another example, backward design 
strategic scaffolding used in conjunction with reflection rubrics helped high school 
science students judge the quality with which they collected data and other informa- 
tion, as well as the quality of their research reports and their peer reviews (Deters, 
2008). Backward design scaffolding by itself led to a statistically significant and 
substantial effect on lab report quality, but when reflective prompts were used in 
conjunction with backward design scaffolding, there was no difference between the 
performances of experimental and control students (Deters, 2008). 


5.2.4 Motivation Scaffolding 


Motivation scaffolds primarily aim to enhance students’ academic motivation toward 
the target content, defined as their willingness to deploy effort to carry out learning 
tasks (Tuckman, 2007; Wigfield & Eccles, 2000). This can be done through one of 
the following processes or a combination thereof: enhancing students’ (a) expectan- 
cies for success, (b) perceptions of value in the completion of the target task, (c) 
perceptions of self-determination of behavior, (d) perceptions of mastery goals, (e) 
abilities to regulate academic emotions, and (f) perceptions of belongingness (Bel- 
land, Kim, et al., 2013). Strategies to do so include establishing attainment value, 
supporting productive attribution, and promoting the perception of optimal challenge 
(Belland, Kim, et al., 2013). Scaffolds have helped promote expectancy for success 
through inviting students to reflect on the efficacy of strategies (Davis & Linn, 2000; 
Herrenkohl & Cornelius, 2013). In addition, providing attributional feedback that 
guides middle school students to attribute failure to lack of effort and success to good 
strategy use has been found to lead to stronger motivation and self-concept among 
experimental students than among control students (Dresel & Haugwitz, 2008). Re- 
searchers deploy motivation scaffolds to increase engagement in the target content 
(Rienties et al., 2012) and to raise academic achievement (Belland, Kim, et al., 2013). 

Historically, most designers aimed to create computer-based scaffolding that pro- 
vided cognitive or motivational support, despite the importance of the integration 
of these two types of support (Belland, 2014; Belland, Kim, et al., 2013; Rienties 
et al., 2012). This approach leaves it entirely to one-to-one or peer scaffolding to 
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provide the form of support that computer-based scaffolding does not. Motivation 
can make a big difference in students’ performance in academic tasks, including the 
type of high-level tasks with which scaffolding is used (Belland, Kim, et al., 2013; 
Bereby-Meyer & Kaplan, 2005; Brophy, 1999; Giesbers, Rienties, Tempelaar, & 
Gijselaers, 2013; Perkins & Salomon, 2012). Expecting all cognitive support to be 
provided by computer-based scaffolds, and all motivational support by one-to-one 
scaffolding, or vice versa, is not likely the best choice. That is, in a typical class- 
room, there is one teacher, and that one teacher cannot work with all students at all 
times (Belland, 2012; Belland, Burdo, & Gu, 2015; Hogan & Pressley, 1997; Saye 
& Brush, 2002). 

An alternative to assigning one scaffolding function to one-to-one scaffold- 
ing and another scaffolding function to computer-based scaffolding is to design 
scaffolding systems to provide redundancy in support such that students receive 
all needed support even if the teacher needs to work one-to-one with a struggling 
small group for an extended period of time (Puntambekar & Kolodner, 2005; Ta- 
bak, 2004). Such a scaffolding system can include computer-based scaffolding, peer 
scaffolding, and one-to-one scaffolding, and redundancy can be across and within 
scaffolding types (Belland, Gu, et al., 2013; Puntambekar & Kolodner, 2005). Pro- 
viding such redundancy may allow students to be more likely to benefit from scaf- 
folding support at the time they need it than if such support were only provided by 
one scaffolding mode (Puntambekar & Kolodner, 2005). 


5.2.5 Results from the Meta-Analysis 


The meta-analysis included 227 outcomes of conceptual scaffolding (g=0.48), 28 
outcomes of metacognitive scaffolding (g=0.42), 75 outcomes of strategic scaf- 
folding (g=0.44), and 3 outcomes of motivation scaffolding (g=0.41; Note: to be 
included, outcomes needed to be cognitive) (see Table 5.1; Belland, Walker, Kim, 
& Lefler, In Press). There were no statistically significant differences among 
scaffolding types, p>0.05. One interesting aspect of this finding is that it suggests 
that metacognitive scaffolding leads to strong learning outcomes. Metacognitive 
scaffolding has often been criticized, in part due to observations in the literature 
that students often do not use it (Belland, Glazewski, & Richardson, 2008; Oliver & 
Hannafin, 2000). But results suggest that it is as effective as other major scaffold- 
ing types. This provides a preliminary suggestion that rather than attempt to choose 


Table 5.1 Table of results of moderator analyses on the effect of type of scaffolding intervention 
on cognitive outcomes 


[95 % Confidence interval 


Level |n outcomes | Effect size estimate | Lower limit | Upper limit . 
Conceptual scaffolds | 227 [0.48 | 041 (0.54 
Metacognitive scaffolds | 28 0.42 | 0.23 | 0.60 
Motivation scaffolds | 3 [0.41 -0.02 0.85 


Strategic scaffolds | 75 [0.44 | 0.36 [0.53 


114 5 Computer-Based Scaffolding Strategy 


a scaffolding type that is most effective and design accordingly, it is better to first 
decide on the nature of support students need, and then design the scaffolding sup- 
port accordingly. 


5.3 Context Specificity 


In this section, I first describe what context specificity is with regard to scaffolding. 
Then, I address what the meta-analysis indicates about differences in effect sizes 
between context-specific and generic scaffolding. 


5.3.1 What It Is 


There has been much debate as to whether it is important to embed context-specific 
support in computer-based scaffolds. Much of this has to do with long-standing 
debates as to whether problem-solving skills are generic or context-specific; for 
an overview of the latter debate, see Perkins and Salomon (1989) and Schunn and 
Anderson (1999). Within one-to-one, teacher scaffolding, this question would be of 
little importance, as teachers can dynamically determine if such contextual support 
was needed. But given that computer-based scaffolding is designed before students 
use it, it is an important question to consider (Akhras & Self, 2002; Belland, 2014). 

Computer-based scaffolding can be tailored to specific content or designed to 
be more generic in its approach (Belland, 2014; McNeill & Krajcik, 2009). For 
example, ExplanationConstructor was designed to be context specific (Sandoval & 
Reiser, 2004). Thus, all of its prompts included specific content related to the prob- 
lem that students were addressing—microevolution among ground finches in the 
Galapagos Islands. As an example of a generic scaffold, the Collaborative Concept 
Mapping Tool was designed to facilitate groups’ shared creation of concept maps in 
conjunction with units of different topics (Gijlers, Saab, Van Joolingen, de Jong, & 
Van Hout-Wolters, 2009). 

One of the arguments advanced for using context-specific scaffolding is the idea 
that problem-solving skills are usually context bound, which emerged in part as 
a reaction against the practice of developing problem-solving heuristics based on 
such games as Towers of Hanoi and Missionaries and Cannibals (Perkins & Salo- 
mon, 1989). However, there is a strong evidence that problem-solving involves a 
mix of domain-specific and generic skills (Klahr & Simon, 1999; Molnar, Greiff, & 
Csapó, 2013; Perkins & Salomon, 1989; Schunn & Anderson, 1999). In this way, it 
is important to consider the nature of the subskill that one wishes to support through 
scaffolding in order to decide whether to use context-specific or generic scaffolding 
(Belland, Gu, et al., 2013). 

There is not a large amount of research that directly compares the effectiveness of 
generic and context-specific scaffolds. A pilot meta-analysis found no difference in 
effect sizes between context-specific and generic scaffolds (Belland, Walker, Olsen, 
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Table 5.2 Results of moderator analyses on the effect of type of context specificity on cognitive 
outcomes 
| 95% Confidence interval 


Level __ |noutcomes | Effect size estimate | Lower limit | Upper limit 
Context-specific |273 | 0.46 0.41 | 0.52 
Generic | 60 (0.48 [0.35 [0.60 


& Leary, 2015). However, there is some evidence about specific questions related 
to context-specific and generic scaffolding. For example, there is some evidence 
that generic prompts for reflection promote better science learning among middle 
school students than context-specific ones (Davis, 2003). However, reflection is 
not the only process supported by scaffolding. There is also evidence that synergy 
is promoted when teachers provide one-to-one scaffolding from the perspective of 
a generic argumentation framework and computer-based scaffolds provide context- 
specific argumentation scaffolding, thereby maximizing middle school students’ 
learning of argumentation (McNeill & Krajcik, 2009). 

Rather than simply declaring that generic or context-specific scaffolding is the 
best, a better approach may be to consider how to combine context-specific and 
generic scaffolding, as well as one-to-one and computer-based scaffolding, accord- 
ing to the types of skills to be supported and the inherent strengths of a generic ap- 
proach versus that of a context-specific approach, and that of a one-to-one versus a 
computer-based approach (Belland, Gu, et al., 2013). That is, one can consider how 
to create a portfolio of generic and context-specific scaffolding that optimally sup- 
ports student learning and performance. 

Beyond the suitability of scaffolding strategies for supporting specific skills, 
there are considerations regarding scalability. If a scaffold is entirely context-specif- 
ic, then all of its instructional messages are inextricably tied to the specific content 
with which students are working in the target unit (Belland, Gu, et al., 2013). As 
such, the scaffold can only be used in the context of the target unit. Generic scaf- 
folds use language that is not tied to the target unit such that they can be used in 
conjunction with other units. 


5.3.2 Results from Meta-Analysis 


The meta-analysis included approximately 4.5 times as many outcomes from studies 
that investigated context-specific scaffolds (n=273) than from studies that inves- 
tigated generic scaffolds (n=60) (see Table 5.2; Belland, Walker, Kim, & Lefler, 
In Press). When one considers that 82% of included outcomes in the meta- 
analysis were associated with context-specific scaffolding, it seems clear that scaf- 
folding designers are choosing to design context-specific scaffolding more often 
than generic scaffolding. This may be based on the idea that the type of strategies 
that one tried to promote through scaffolding (e.g., problem-solving strategies) are 
inherently context-specific and cannot be performed or learned sufficiently without 
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an adequate base of conceptual knowledge. But there was no significant difference 
between the average effect sizes when generic scaffolding (g=0.48) and context- 
specific scaffolding (g=0.46) were used, p=0.778. This suggests that arguments 
that problem-solving and other strategies are context-specific and need to be sup- 
ported by context-specific scaffolding are not supported by the corpus of empirical 
evidence on computer-based scaffolding in STEM education, or at least that which 
met the inclusion criteria (namely, met scaffolding definition, had an experimental 
and a control group, and contained sufficient information to calculate an effect size). 
That is, students seem to do equally well whether domain knowledge is embedded 
in the scaffolding or not. Thus, scaffolding designers can choose to use generic or 
context-specific scaffolding based on a determination of which strategy works best 
under the constraints of the learning context, rather than a consideration of which 
strategy is the most effective (Belland, Gu, et al., 2013). 


5.4 Customization Presence or Absence 


One of the biggest sticking points as the metaphor of scaffolding was applied to 
computer-based tools was the issue of contingency—namely, whether scaffolding 
was added, faded, or added and faded based on an estimation of the current ability of 
the student. As computer-based scaffolding was introduced, much lacked anything 
in the way of contingency, leading some authors to question whether the tools could 
be called scaffolding at all (Pea, 2004; Puntambekar & Hiibscher, 2005). Indeed, Pea 
(2004) noted that such tools may be better described as part of distributed cognition, 
defined as a system in which information and an executive function are distributed 
among various individuals and tools such that no one entity carries out the entire ex- 
tent of cognition required by the task (Belland, 2011; Giere, 2006). Most such argu- 
ments have been voiced by researchers from the activity-theory- and knowledge-in- 
tegration-informed scaffolding traditions. This is perhaps because fading and adding 
is consistently applied in ACT-R-informed scaffolding (Koedinger & Aleven, 2007). 

A closer look at the nature of scaffold fading, adding, and fading/adding is war- 
ranted. Fading refers to gradually removing support as students gain skill (Collins 
et al., 1989; Wood, Bruner, & Ross, 1976). One can base fading on dynamic assess- 
ment of students’ capabilities, though fading in much computer-based scaffolding 
is based on self-selection and fixed intervals. Fading can involve but is not limited 
to gradually transitioning students to a less supportive/directive form of support, 
lessening the frequency of prompts, and lessening the specificity of feedback. For 
example, one scaffold for high school students progressed from providing sentence 
starters in the body of a text box to a simple prompt to formulate sources to no 
prompt at all; this progression happened on a fixed schedule (Raes, Schellens, De 
Wever, & Vanderhoven, 2012). 

Adding support refers to increasing the strength or frequency of support as per- 
formance indicators show that students need more support (Koedinger & Alev- 
en, 2007). As with fading, this should be implemented on the basis of dynamic 
assessment, though it is often based on self-selection (Koedinger & Aleven, 2007). 
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Table 5.3 Results of moderator analyses on the effect of customization on cognitive outcomes _ 
| 95% Confidence interval 


Level |n outcomes Effect size estimate | Lower limit | Upper limit 
Fading [12 [0.62 [0.38 [0.87 
Adding |62 [0.46 0.35 (0.56 
Fading/adding | 43 (0.50 | 0.36 (0.63 
None 216 (0.46 [0.39 [0.52 


Adding can involve providing more directive support, providing additional feed- 
back of a different nature, and increasing the frequency of prompts. For example, 
the Mobile Knowledge Constructor invited students to find a plant in a garden (Chu, 
Hwang, & Tsai, 2010). It then asked questions about features of the target plant. If 
students answered incorrectly, it guided them to another plant that has the mistaken 
feature. After studying the new plant, students needed to answer the question they 
missed again. 

Fading and adding is simply the combination of fading and adding within the 
same scaffolding treatment. As with fading and adding, fading/adding should be 
performed on the basis of dynamic assessment. Accordingly, fading occurs when 
performance indicates that students are gaining sufficient skill to perform the target 
task independently, whereas adding occurs when students are not on track to im- 
prove as rapidly as desired. For example, a scaffolding system broke content to be 
learned into different blocks (S. Li, 2001). For each block, there were four levels of 
support possible: no support, provide hint, provide example, and provide answer. 
Students started out at the hint level. In the system-controlled version, if they an- 
swered correctly, they would be moved down to no support. If they answered incor- 
rectly, then they would be provided an example, and so on. 

No fading/adding means that there is no customization of scaffolding. In other 
words, scaffolding is the same throughout students’ engagement with the central 
problem. Researchers often argue that not-fading/adding can lead to overscripting, 
a situation in which scaffolding is provided when it is in fact not needed, thereby 
conflicting with existing mental models of how to address the targeted problem 
(Dillenbourg, 2002). This in turn is said to lead to weaker learning outcomes. 


5.4.1 Results from Meta-Analysis 


It makes sense to take a step back from the theoretical arguments to see if scaffold 
customization actually impacts cognitive outcomes. The scaffolding meta-analy- 
sis covered outcomes of scaffolds that incorporated several variations of contin- 
gency—fading (n=12), adding (n=62), fading and adding (n=43), as well as no 
fading or adding (n=216) (see Table 5.3) (Belland et al., In Press). Thus, 
the majority of outcomes were associated with no fading or adding (64.9%). Of 
the included outcomes, 16.5% were associated with scaffolding that incorporated 
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fading in some way, either just fading, or fading and adding. This is close to what 
was found in the review of scaffolding research by Lin et al. (2012), who found that 
9.3% of the reviewed studies incorporated fading. And it is generally consistent 
with the lamentations of scaffolding scholars (Pea, 2004; Puntambekar & Hiibscher, 
2005). This appears to confirm that fading is rarely incorporated in scaffolding. 

There was no significant difference in cognitive outcomes among the differ- 
ent contingency types. This is interesting in that authors often lament the lack of 
attention to scaffolding customization. But the results indicate that the presence or 
the type of scaffolding customization does not influence cognitive outcomes. Sim- 
ply put, from a cognitive outcome standpoint, incorporating fading, adding, or fad- 
ing/adding made no difference. This suggests that researchers might be best served 
considering other scaffolding factors in their quest to maximize student learning 
from scaffolding. 


5.5 Customization Basis 


While in the original scaffolding definition, customization was based on a teacher’s 
assessment of students’ performance indicators (Wood et al., 1976), customization 
of computer-based scaffolding has not always been performance based. When scaf- 
folding is customized based on performance indicators, the scaffolding engages in 
dynamic assessment of student performance. For example, students may need to 
complete a quiz. Based on their score, scaffolding is customized. This is often done 
in intelligent tutoring systems (Koedinger & Corbett, 2006). 

Other strategies used as the basis of the customization of computer-based scaf- 
folding included setting scaffolding to reduce in strength according to fixed time 
intervals (fixed fading) (Dori & Sasson, 2008; McNeill, Lizotte, Krajcik, & Marx, 
2006; Philpot, Hall, Hubing, & Flori, 2005) or when students click a button (self- 
selected fading) (Clark, Touchman, Martinez-Garza, Ramirez-Marin, & Skjerping 
Drews, 2012; Metcalf, 1999; Renkl, 2002). Customization based on fixed time in- 
tervals means that the scaffold designer determines time intervals after which scaf- 
folding should be faded or added. Once the time interval is passed, the scaffolding 
would be added or faded automatically. Self-selection means that a button is pro- 
vided with which students can request hints (adding) or request that scaffolding be 
removed (fading). For example, adding scaffolding (hints) has also been linked to 
pressing buttons in intelligent tutoring systems (Koedinger & Aleven, 2007), and 
fading has been controlled by students who press a button indicating that they per- 
ceive that they do not need the scaffolding any longer (Metcalf, 1999). The rationale 
for the use of self-selection in adding in intelligent tutoring systems is to avoid un- 
productive struggle, and it is thought that students can recognize that they are strug- 
gling too much (Koedinger & Aleven, 2007). Similarly, in self-selected fading, it is 
thought that learners can accurately gauge the extent to which they need scaffolding 
assistance at a given point in a learning task (Metcalf, 1999). This relies on learners 
to make good instructional decisions, which they often struggle to do (Williams, 
1996). Furthermore, fixed and self-selected customization does not appear to fit the 
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original definition and may not have been performed on the basis of performance 
characteristics (Belland, 2011). 

Sometimes, scaffolding can be customized on the basis of performance indica- 
tors and self-selection. For example, students may self-select a level of scaffolding 
that they want before engaging with the scaffold; as they engage in the system, the 
system may provide feedback and suggestions to adjust the self-selected scaffolding 
level on the basis of performance characteristics (Cheng et al., 2009). Intelligent 
tutoring systems often provide feedback on the basis of performance indicators, but 
students can also request more help by clicking a hint button. If the first hint does 
not help enough, the student can click the hint button again to get a more detailed 
hint, until eventually he/she is given the solution (Koedinger & Aleven, 2007). In- 
telligent tutoring systems based on the ACT-R model of cognition guide students 
through a task using several strategies, including providing choices on what meth- 
ods to use to solve the target problem, feedback on what students do, and hints on 
how to accomplish certain steps (Koedinger & Aleven, 2007; Koedinger & Corbett, 
2006; VanLehn, 2011). According to ACT-R, complex cognitive domains can be 
seen as a set of production rules and declarative knowledge, and such production 
rules can be learned independently (Anderson, 1983; Anderson et al., 1997; Koed- 
inger & Aleven, 2007). Hints are designed to reduce the amount of unproductive 
practice in which students engage, which ACT-R posits as an impediment to learn- 
ing (Anderson, 1990; Anderson et al., 1997). Sometimes, hints are requested by 
students, and sometimes they are provided based on the intelligent tutoring system’s 
estimation of student ability. Intelligent tutoring systems also keep track of stu- 
dents’ abilities through knowledge tracing (Koedinger & Aleven, 2007; Koedinger 
& Corbett, 2006). In this way, they estimate whether students know or do not know 
the production rule under study. Through knowledge tracing, an intelligent tutoring 
system can estimate when a student is ready to proceed to the next unit and select 
problems of appropriate difficulty (Koedinger & Aleven, 2007; Koedinger & Cor- 
bett, 2006). It can also determine when a student needs more or less support, and 
adjust the support accordingly. 

Preliminary meta-analyses of scaffolding indicated that fixed fading led to an av- 
erage effect that was not significantly different from zero (Belland, Walker, Kim, & 
Lefler, 2014; Belland, Walker, et al., 2015). Linking customization to self-selection 
poses challenges as well. In the case of intelligent tutoring systems, hints usually 
become successively more detailed/supporting, causing some students to game the 
system by pressing the button multiple times until they get the answer (Koedinger & 
Aleven, 2007). Furthermore, computer-based scaffolding rarely incorporates feed- 
back (Belland, 2014). 


5.5.1 Results from Meta-Analysis 


Again, taking a step back from the theoretical arguments, it is important to examine 
whether the basis by which scaffolding is faded, added, or faded and added influ- 
ences cognitive outcomes. Of the outcomes in which scaffolding customization was 


120 5 Computer-Based Scaffolding Strategy 


Table 5.4 Results of moderator analyses on the effect of customization schedule on cognitive 
outcomes 

l | 95% Confidence interval 
Level |n outcomes | Effect size estimate _ | Lower limit | Upper limit 


Performance-based | 63 0.47 | 0.35 | 0.60 
Fixed | 13 | 0.62 | 0.42 | 0.82 
Self-selected | 41 [0.45 [0.35 | 0.55 
None | 216 0.45 [0.39 | 0.52 


present, 53.8% involved performance-based customization, 35% involved self- 
selection, and 11.1% involved fixed customization (See Table 5.4; Belland et al., 
In Press). Of note, many scaffolding interventions that incorporated perfor- 
mance-based customization were embedded in intelligent tutoring systems. In such 
cases, even though there was often both performance-based fading and self-selected 
adding, the scaffold was classified as performance-based since the performance- 
based customization would always be present and theoretically always happen, 
while self-selected adding would only happen if students clicked the hint button. 
Future research may attempt to disentangle such combinations of scaffolding bases 
to tease apart the effect of these different scaffolding components. However, this 
would be difficult as one would likely need to be able to attribute outcomes to 
specific scaffolding components for which customization was performance-based, 
and other outcomes to other outcomes that were self-selected. The inclusion of such 
outcomes that can be easily attributed to separate scaffolds is quite rare. 

There were no statistically significant differences among the scaffolding cus- 
tomization bases. This means that there were no differences between performance- 
based customization, fixed customization, self-selected customization, and no cus- 
tomization. This largely flies in the face of the generally accepted consensus among 
scaffolding scholars that scaffolding customization is better than no scaffolding 
customization, and that performance-based customization is the best of all. From 
a statistical standpoint, there was no difference in cognitive outcomes. This is very 
interesting. Of course, further research is needed to understand the role of scaf- 
folding customization and scaffolding customization bases in STEM learning. For 
example, only cognitive outcomes were included; there may be differences in terms 
of motivation or self-direction. This finding conflicts with the findings from a pilot 
scaffolding meta-analysis that indicated that when scaffolding was not faded, effect 
sizes were higher than when scaffolding was faded on a fixed schedule (Belland, 
Walker, et al., 2015). In yet another prior scaffolding meta-analysis, fixed fading led 
to an effect size that was not significantly greater than zero, while not-fading led to 
an effect size that was significantly greater than zero (Belland et al., 2014). 
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Chapter 6 
Conclusion 


Abstract In this chapter, I conclude this book on computer-based scaffolding in 
science, technology, engineering, and mathematics (STEM) education. I note the 
overall effect size point estimate for scaffolding—g=0.46—and compare that to 
other effect size estimates in the literature. I summarize the wide variation in con- 
texts in which and learner populations among which scaffolding is used, as well 
as note the characteristics along which the magnitude of scaffolding’s impact does 
not vary—contingency, generic versus context specific, and intended learning out- 
come—as well as characteristics along which it does—problem-centered model 
with which scaffolding is used, and grade level and learner characteristics. I also 
note areas in which more research is needed—motivation scaffolding, scaffolding 
for students with learning disabilities, and scaffolding in the context of project- 
based and design-based learning. 


Keywords Content learning - Context specificity - Problem-centered instruction - 
Scaffolding customization : Scaffolding strategy - STEM disciplines 


6.1 Overall Implications 


Despite the attempt by Kirschner, Sweller, and Clark (2006) to posit problem-cen- 
tered instructional approaches as failures due to their purported lack of instructional 
guidance, it has been seen in this book that problem-centered instruction paired 
with computer-based scaffolding is quite effective in promoting strong cognitive 
outcomes. Scaffolding leads to effects that were significantly greater than zero and 
practically important across the concept, principles, and application assessment 
levels (Belland, Walker, Kim, & Lefler, In Press; Sugrue, 1995). As strength 
across such a wide range of assessment levels was not found in meta-analyses of 
problem-centered instructional approaches by themselves (e.g., Albanese & Mitch- 
ell, 1993; Gijbels, Dochy, Van den Bossche, & Segers, 2005; Walker & Leary, 
2009), one can conclude that it is the instructional support of computer-based scaf- 
folding that leads to the strong outcomes. 
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Scaffolding used in the context of problem-centered instruction led to an average 
effect size of g=0.46 on cognitive outcomes. This is in line with results from prior 
meta-analyses, which indicated overall effect sizes of g=0.53 (Belland, Walker, 
Olsen, & Leary, 2015) and g=0.44 (Belland, Walker, Kim, & Lefler, 2014) for 
computer-based scaffolding in science, technology, engineering, and mathematics 
(STEM) education. It is below the effect size estimate for step-based intelligent 
tutoring systems (ES=0.76) found in a recent review (VanLehn, 2011), but this 
is to be expected as our review covered a much wider variety of scaffolding treat- 
ments. Briefly, computer-based scaffolding has a substantial impact on cognitive 
outcomes. This is consistent with prior research (Alfieri, Brooks, Aldrich, & Te- 
nenbaum, 2011; Belland et al., 2014; Belland, Walker, et al., 2015; Dochy, Segers, 
Van den Bossche, & Gijbels, 2003; Gijbels et al., 2005; Hmelo-Silver, Duncan, & 
Chinn, 2007; Kuhn, 2007; Schmidt, van der Molen, te Winkel, & Wijnen, 2009; 
Strobel & van Barneveld, 2009; Swanson & Lussier, 2001; Walker & Leary, 2009) 
and also reflects well on the considerable investment that has been made developing 
and studying scaffolding. 

Although the intended learning outcomes of computer-based scaffolding in- 
clude both content-learning and the development of higher-order thinking skills, 
it is worthwhile to compare its average effect size with that of a wider range of 
instructional interventions designed to enhance critical thinking skills, and educa- 
tional technology interventions as a whole. Computer-based scaffolding’s effect 
(g=0.46) is greater than the average effect size of educational technology inter- 
ventions designed to support direct instruction (ES=0.31) found in a synthesis of 
meta-analyses of educational technology interventions conducted over the course 
of 40 years (see Fig. 6.1; Tamim, Bernard, Borokhovski, Abrami, & Schmid, 2011). 
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Fig. 6.1 Computer-based scaffolding’s effect size estimate as compared to that of related educa- 
tional technology interventions 
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It is also higher than the effect size estimates of interventions designed to increase 
critical thinking abilities: ES=0.195 (Niu, Behar-Horenstein, & Garvan, 2013) and 
ES=0.341 (Abrami et al., 2008). It is also higher than the average effect size of edu- 
cational technology applications designed for mathematics education (ES=0.13; 
Cheung & Slavin, 2013) and that of educational technology applications designed 
for reading instruction (ES=0.16; Cheung & Slavin, 2012) found in recent reviews. 
Furthermore, the average effect size for computer-based scaffolding is higher than 
the median effect size among meta-analyses of interventions in psychological re- 
search (g=0.324; Cafri, Kromrey, & Brannick, 2010). Briefly, the magnitude of 
the effect of computer-based scaffolding on cognitive outcomes is substantial when 
compared to instructional interventions that seek to influence similar outcomes, and 
also compared to other educational technology interventions and interventions in 
psychological research. 

Computer-based scaffolding includes a wide variation of interventions, ranging 
from scaffolding embedded in intelligent tutoring systems, which contain all mate- 
rial to be encountered by students and which fade scaffolding based on a compari- 
son of student performance with a model of an idealized student and allow students 
to add scaffolding by clicking a hint button (Koedinger & Corbett, 2006; Means 
& Gott, 1988) to tools used when investigating problems in the outside world that 
often do not involve fading or adding (Pea, 2004; Reiser, 2004). This large varia- 
tion in scaffolding can be traced to the different theoretical frameworks (i.e., ac- 
tivity theory (Leont’ev, 1974; Luria, 1976; Vygotsky, 1978), Adaptive Character 
of Thought—Rational (ACT-R; Anderson, 1983; Anderson, Matessa, & Lebiere, 
1997), and knowledge integration (Linn, 2000; Linn, Clark, & Slotta, 2003)) that 
were integrated into the relatively atheoretical initial conceptualization of scaf- 
folding (Wood & Wood, 1996). Each of these theoretical frameworks has different 
views on the nature of learning and the goal of instruction. Still, the characteristics 
on which scaffolding informed by these different theoretical frameworks varies— 
contingency, generic versus context-specific, and intended learning outcome—did 
not explain any significant differences in cognitive outcomes. This suggests that the 
effect of scaffolding on cognitive learning outcomes is robust to different intended 
learning outcomes and the choice of whether or not to embed content knowledge in 
scaffolding, and is largely robust to the presence or absence of scaffolding custom- 
ization as well as customization bases. 


6.2 How the Meta-Analysis Responds to Persistent 
Debates in the Scaffolding and Problem-Centered 
Instruction Literature 


This book presents some interesting answers to questions regarding scaffolding 
customization, the role of context-specific information in scaffolding, and whether 
scaffolding should be geared toward promoting enhanced content learning or high- 
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er-order thinking abilities. I certainly do not consider such questions to be answered 
definitively, as there is much to be learned when considering these findings along- 
side findings of empirical studies that were not eligible for inclusion in the meta- 
analysis. Such may be accomplished through the use of meta-synthesis (Bondas & 
Hall, 2007; Finfgeld, 2003; Thorne, 2004) and other synthesis efforts. Such further 
work can help to further address these questions and help scaffolding developers 
and researchers learn the most effective scaffolding strategies. 


6.2.1 Scaffold Customization 


Scaffolding scholars from the various scaffolding theoretical traditions have long 
posited scaffolding customization as a necessary attribute of scaffolding (Collins, 
Brown, & Newman, 1989; Pea, 2004; Puntambekar & Hiibscher, 2005). This was 
clearly an important part of the original scaffolding definition; scaffolding custom- 
ization unfolded as teachers dynamically assessed students’ current abilities and 
adjusted the support that was given accordingly. Scholars from the intelligent tutor- 
ing systems tradition have long called for the use of fading and adding (Aleven, 
Stahl, Schworm, Fischer, & Wallace, 2003; Koedinger & Aleven, 2007), while 
scholars from the knowledge integration and activity theory traditions have called 
for the use of fading (Collins et al., 1989; McNeill, Lizotte, Krajcik, & Marx, 2006; 
Pea, 2004; Puntambekar & Hiibscher, 2005). Indeed, some scholars suggested that 
interventions that do not include fading cannot be called scaffolding (Pea, 2004; 
Puntambekar & Hiibscher, 2005). The count of outcomes in which scaffolding was 
faded or added versus when scaffolding was neither added nor faded indicated that 
the majority of outcomes were associated with no fading or adding (64.9%), which 
is consistent with prior research (Lin et al., 2012; Pea, 2004; Puntambekar & Hüb- 
scher, 2005). But the meta-analysis suggests that scaffold customization does not 
influence cognitive outcomes. Further research is needed to fully understand the 
role of scaffold customization in promoting learning. 

Cognitive outcomes are only one way to characterize the success (or lack there- 
of) of an instructional intervention/feature. Other ways include attitudinal and af- 
fective outcomes and the capacity of the intervention to foster transfer, neither of 
which were the focus of the underlying meta-analysis of this book. Indeed, one of 
the arguments in favor of fading holds that providing scaffolding support when it 
is not needed can undermine motivation, thereby decreasing learning and perfor- 
mance (Dillenbourg, 2002). Motivation is a very important influence on learning 
(Belland, Kim, & Hannafin, 2013; Fredricks, Blumenfeld, & Paris, 2004; Wigfield 
& Eccles, 2000), and so investigating the influence of scaffolding customization 
(or lack thereof) on motivation, and consequently on achievement, is important and 
warrants future research. 

One can also examine the extent to which scaffolding leads to transfer, including 
students’ preparation for future learning (Bransford & Schwartz, 1999) and their 
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ability to recognize similarities between the learning context and new contexts in 
which the learning can be applied (Lobato, 2003). Transfer is clearly an important 
goal of problem-centered instruction and forms one of the key pillars in the ra- 
tionale for such approaches (Hmelo-Silver, 2004). Does scaffolding customization 
influence transfer? This is an empirical question that warrants future research. 


6.2.2 Problem-Centered Instruction and Content Learning 


One of the persistent criticisms of problem-centered instructional models is that 
they do not do a good job in promoting concept-level learning (Kirschner et al., 
2006). The thinking goes that problem-based learning does a better job than lecture 
at promoting learning at the principles and application levels but does not do as well 
at promoting concept-level learning (Albanese & Mitchell, 1993; Berkson, 1993). 
This is borne out in most meta-analyses of problem-based learning that break learn- 
ing down by assessment level (Albanese & Mitchell, 1993; Gijbels et al., 2005; 
Kalaian, Mullan, & Kasim, 1999; Vernon & Blake, 1993; Walker & Leary, 2009), 
and has been found to be consistent outside of medical education (Walker & Leary, 
2009). One exception to this trend is that problem-based learning seems to tend to 
lead to stronger long-term concept learning than lecture (Dochy et al., 2003; Strobel 
& van Barneveld, 2009). 

One review found mixed results on inquiry-based learning’s influence on con- 
cept learning, finding that student concept learning was predicted by the extent to 
which students needed to think actively and draw conclusions from data, rather 
than by the simple use of inquiry-based learning (Minner, Levy, & Century, 2010). 
Another review indicated that when inquiry-based learning aims at promoting epis- 
temic and conceptual learning, effect sizes tend to be quite low (ES=0.19) as com- 
pared to studies that focused squarely on epistemic learning goals (ES=0.75) or 
on procedural, epistemic, and social goals (ES=0.72; Furtak, Seidel, Iverson, & 
Briggs, 2012). 

In this meta-analysis, scaffolding used in the context of problem-centered in- 
structional models led to average effect sizes at the concept, principles, and applica- 
tion levels of g=0.40, g=0.51, and g=0.44, respectively. These are all substantial 
effect sizes and mean that scaffolding leads to strong learning outcomes across the 
three assessment levels (Sugrue, 1995). The findings suggest that by employing 
computer-based scaffolding along with problem-centered instructional models, one 
can erase the former liability of problem-centered instructional models—poor con- 
cept learning. This makes sense when one considers that inquiry-based learning led 
to strong effect sizes on content learning when students needed to engage in active 
thinking (Minner et al., 2010). Scaffolding promotes active thinking on the part of 
students, and often encourages them to draw conclusions from data (Belland, 2014; 
Quintana et al., 2004; Reiser, 2004). 
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6.2.3 Context Specificity 


Much work on scaffolding in science has focused on context-specific scaffolding 
due to thoughts that (a) scientific problem-solving is highly context specific (Abd- 
El-Khalick, 2012; McNeill & Krajcik, 2009; Perkins & Salomon, 1989) and (b) 
any problem-solving strategy that involves any domain-specific knowledge is itself 
domain specific (Smith, 2002). Furthermore, there are arguments that one does not 
need to teach generic skills, based on a premise that individuals will simply pick 
up the generic skills they need through everyday life (Tricot & Sweller, 2013). That 
the majority of computer-based scaffolding is context specific was confirmed by 
the fact that 82% of the outcomes included in the meta-analysis were associated 
with context-specific scaffolding. The arguments above against scaffolding generic 
processes appear to be tenuous arguments, and one would be better served to look 
at the empirical evidence to decide whether generic or context-specific scaffolding 
is more effective. 

Much evidence indicates that scientific problem-solving in fact incorporates a 
mix of domain-specific and generic processes (Klahr & Simon, 1999; Molnar, Gre- 
iff, & Csapó, 2013; Perkins & Salomon, 1989; Schunn & Anderson, 1999). For ex- 
ample, evaluating sources can involve domain-specific knowledge, but the underly- 
ing strategy can be considered generic (Smith, 2002). There is not a large amount of 
empirical work addressing the relative effectiveness of generic and context-specific 
scaffolding. But we addressed it in the meta-analysis, finding no differences in cog- 
nitive outcomes between generic and context-specific scaffolding. Therefore, one 
may envision the need for a mix of generic and context-specific scaffolding that can 
allow the strengths of each scaffolding type to complement each other (Belland, Gu, 
Armbrust, & Cook, 2013). 


6.2.4 Higher-Order Thinking Skills Versus Content Knowledge 


Scaffolding has been used to promote the development of higher-order thinking 
skills (Belland, Glazewski, & Richardson, 2011; Belland, Gu, Armbrust, & Cook, 
2015; Kim & Hannafin, 2011) and enhanced content knowledge (Chang & Linn, 
2013; Davis & Linn, 2000)—two seemingly disparate instructional goals. These 
differences in instructional goals can be linked to differences in the theoretical bases 
to which scaffolding is tied. These differences in theoretical bases lead to real dif- 
ferences in scaffolding strategies, such as the use of adding and fading (Koedinger 
& Aleven, 2007) versus fading (Pea, 2004; Puntambekar & Hiibscher, 2005), dif- 
ferences in intended learning outcomes, and differences in contexts of use. Such a 
disparity in intended learning outcome may lead one to think that these are quali- 
tatively different interventions. Yet, the scaffolding definition that noted that scaf- 
folding needs to extend and enhance student abilities as they engage in authentic 
problem-solving was carefully applied. Meta-analysis indicated that the two scaf- 
folding types lead to effect sizes that were statistically the same. 
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6.2.5 Scaffolding Strategy 


Scaffolding can incorporate a variety of approaches according to what processes it 
aims to support in students, including conceptual, strategic, metacognitive, and mo- 
tivational scaffolding (Belland, Kim, et al., 2013; Hannafin, Land, & Oliver, 1999). 
Designers of computer-based scaffolding often chose to support either motivation 
or cognition (Belland, Kim, et al., 2013), and the effectiveness of metacognitive 
scaffolding has often been questioned (Belland, Glazewski, & Richardson, 2008; 
Oliver & Hannafin, 2000). But the meta-analysis indicated that there were no differ- 
ences in cognitive student outcomes on the basis of scaffolding strategy. Certainly, 
further research is needed to ascertain if the integration of support for motivation 
and cognition in the same scaffold leads to stronger learning outcomes than when 
such support is separated. 


6.2.6 Summary 


Briefly, decisions about whether to (a) include context-specific content or not, (b) 
target higher-order thinking abilities or content knowledge, and (c) fade, add, or 
fade and add scaffolding and on what basis can be made without fear of adversely 
impacting learning outcomes. Rather, such decisions can be made in the context of 
learning goals and what is known about the target learner population. And further 
research is needed to determine if these conclusions apply to education areas other 
than STEM. 


6.3 Other Interesting Findings 


6.3.1 Scaffolding’s Effectiveness in Different STEM 
Disciplines 


It was interesting that computer-based scaffolding was equally effective, statisti- 
cally, in science, technology, engineering, and mathematics. This suggests that scaf- 
folding is a highly effective intervention that is appropriate for use with a wide 
range of authentic problems across STEM. Clearly, addressing authentic problems 
is a crucial skill throughout STEM. It would be unwise to think that students will 
automatically have the skills to be able to do so, or that if they learn declarative 
content, they will figure out how to apply the content to authentic problems. Fur- 
thermore, there is a need for more primary research to be done on scaffolding in 
engineering and mathematics education; such further research is needed to obtain a 
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more precise estimate of the effect of scaffolding used in the context of mathematics 
and engineering education. Certainly, computer-based scaffolding would seem to 
fit well with the types of goals that instructors often have in mathematics and engi- 
neering education—to use the tools of the respective disciplines to model and solve 
problems, both through conceptual solutions and the design of products (Brophy, 
Klein, Portsmore, & Rogers, 2008; Carr, Bennett, & Strobel, 2012; Lesh & Harel, 
2003; Schoenfeld, 1985). 


6.3.2 Scaffolding’s Effectiveness by Grade Level 


Next, it was interesting that scaffolding has come to be used at many different edu- 
cational levels, and the largest effect sizes were among graduate and college learn- 
ers. This is indeed a large expansion of an instructional method originally proposed 
to describe how adults could help toddlers learn to construct pyramids with wooden 
blocks. It also brings to light an important consideration that the distance between 
a more capable other and the learner in graduate education is much less than in 
preschool. There is an expectation that preschool students think about problems in 
qualitatively different ways than do adults (Inhelder & Piaget, 1955), and so the 
metaphor of scaffolding in which a more capable other extends and enhances stu- 
dent cognition makes intuitive sense. But the hope is that graduate students gradu- 
ally begin to think about pertinent problems in the same general manner as their 
professors. In this way, it may be difficult to apply the scaffolding metaphor in an 
intuitive manner to graduate education. Further research is needed to explore the 
role of scaffolding in graduate education and how it differs from scaffolding used in 
the context of other education levels. 


6.4 Directions for Future Research 


This book also suggests directions for future research. In particular, more research 
is needed on motivation scaffolding and scaffolding in the context of design-based 
learning and project-based learning. With the exception of design-based learning, 
these were all associated with particularly large effect size point estimates, but one 
could not have great confidence in the estimates due to a small sample size. For 
design-based learning, the point estimate was low relative to other contexts of use. 

Supporting motivation through scaffolding has often been an afterthought when 
one desires to enhance cognitive skills (Belland, Kim, et al., 2013; Rienties et al., 
2012), and this led us to find only one article that met the inclusion criteria, among 
which was that the student had to measure cognitive outcomes. But its outcomes 
had very large effect sizes. Furthermore, theory suggests that scaffolds that support 
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motivational and cognitive aspects of student work are likely to be more effective 
than scaffolds that focus solely on cognitive factors (Belland, Kim, et al., 2013). 
This may indicate that (a) if more scaffolding is designed to enhance motivation 
alongside cognitive outcomes, one may find very strong effects, and (b) researchers 
would be advised to measure cognitive outcomes resulting from the use of existing 
motivation scaffolds (Brophy, 1999). 

In brief, all of these areas would seem to benefit from more primary research, 
both to improve the precision of estimates of scaffolding’s effect on cognitive out- 
comes and to potentially learn more about a promising way to help students devel- 
op the skills they need to succeed in potentially authentic instructional approaches 
(Belland, 2014; Hmelo-Silver et al., 2007) and the twenty-first century workforce 
(Carnevale & Desrochers, 2003; Gu & Belland, 2015). 

Finally, it is important to investigate the relative impact of scaffolding charac- 
teristics and contexts of use on cognitive outcomes in non-STEM areas (Brush & 
Saye, 2001; Proctor, Dalton, & Grisham, 2007). These are clearly important learn- 
ing outcomes, and enhancing these would not only help students be better prepared 
for careers in STEM but also for the twenty-first century economy in general (Gu 
& Belland, 2015). 

Instructional scaffolding is an effective intervention that can help students per- 
form a half a standard deviation higher than they would have been able to otherwise 
(Belland et al., In Press, 2014; Belland, Walker, et al., 2015; VanLehn, 2011). 
Scaffolding led to effects that were statistically greater than zero across education 
levels ranging from primary to adult. The effect size estimate for middle-level learn- 
ers was lower than that of adult students but still compared favorably to similar 
instructional interventions. Scaffolding also led to effect size estimates that were 
statistically significantly greater than zero across a range of learner populations, 
from underrepresented and underperforming to low income, traditional, and high 
performing. However, underperforming students had a lower effect size estimate 
than traditional students. Scaffolding also had consistently positive effects among 
instructional models with which it is used. Furthermore, scaffolding led to positive 
effect size estimates across science, technology, engineering, and mathematics edu- 
cation. Scaffolding’s sizable impact on cognitive outcomes was largely consistent 
across assessment levels, with the caveat that when learning was assessed at the 
principles level, effect sizes were higher than when assessed at the concept or ap- 
plication level. 

Scaffolding had a positive effect size estimate across customization type (i.e., 
fading, adding, fading/adding, or none), customization basis (i.e., performance 
based, self-selected, and none), or whether or not context-specific information was 
embedded in the scaffolding. There were no significant differences among these 
moderators. Furthermore, the effect size estimate was consistently positive across 
scaffolding intervention types (i.e., conceptual, metacognitive, strategic, and moti- 
vation), and there were no significant differences in this categorization. 
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6.5 Conclusion 


Computer-based scaffolding is a highly effective intervention that leads to strong 
effect sizes that are statistically significantly greater than zero across contexts of 
use, intended learning outcomes, and scaffolding characteristics (Belland et al., 
In Press). Scaffolding is particularly well positioned to help students succeed in 
the problem-centered instructional approaches encouraged by the Next Generation 
Science Standards and Common Core Standards (Achieve, 2013; Krajcik, Codere, 
Dahsah, Bayer, & Mun, 2014; McLaughlin & Overturf, 2012; National Governors 
Association Center for Best Practices & Council of Chief State School Officers, 
2010; National Research Council, 2012). It can do this by extending students’ abili- 
ties in the following areas: argumentation (Belland, 2010; Cho & Jonassen, 2002), 
modeling (Buckland & Chinn, 2010; Fretz et al., 2002), problem-solving (Ge & 
Land, 2003; Raes, Schellens, De Wever, & Vanderhoven, 2012), and forming co- 
herent mental models to describe natural phenomena (Clark & Linn, 2013; Linn, 
2000). As such, computer-based scaffolding is a timely intervention that raises the 
likelihood that problem-centered models will be successful. Research outlined in 
this book can contribute to an understanding of the scaffolding goals, strategies, and 
contexts of use that are associated with the strongest cognitive learning outcomes. 
Results indicate differences in effect sizes based on several characteristics. But 
in most of these cases, effect sizes for the levels of the characteristic that was associ- 
ated with lower effect size estimates were also substantial and significantly greater 
than zero. For example, scaffolding had the highest effect sizes when learning was 
assessed at the principles level, but effect sizes were statistically greater than zero 
and of substantial magnitude across the concept, principles, and application levels. 
Results also help scaffolding researchers learn what scaffolding characteristics do 
not lead to differences in effect sizes—scaffolding customization, generic or context- 
specific nature of support, scaffolding function (e.g., conceptual and strategic), and 
whether scaffolding was designed to enhance content learning or higher-order skills. 
The material covered in this book can be parlayed into stronger scaffolding de- 
signs. Further research should contribute to a greater understanding of the condi- 
tions under which and the strategies with which scaffolding leads to strong learning 
outcomes. Future research should also investigate how to extend scaffolding’s reach 
to benefit underrepresented groups in STEM, a very important goal (Ceci, Williams, 
& Barnett, 2009; Syed, Azmitia, & Cooper, 2011; Thoman, Smith, Brown, Chase, & 
Lee, 2013). This could be pursued through a combination of strategies: look at the 
differences between scaffolds that work well among underrepresented groups, and 
those that are not as effective, examine the literature on designing effective instruc- 
tional supports for members of underrepresented groups (Cuevas, Fiore, & Oser, 
2002; Marra, Peterson, & Britsch, 2008), examine the literature on universal design 
for learning (Rao, Ok, & Bryant, 2014; Scott, Mcguire, & Shaw, 2003), and exam- 
ine whether there are differences in how students from underrepresented groups are 
using scaffolds that could explain lower effectiveness (Belland & Drake, 2013). 
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